-- Logs begin at Wed 2022-05-25 13:04:04 UTC, end at Wed 2022-05-25 13:55:55 UTC. -- May 25 13:37:38.408834 dv-cinder-843278-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 13:37:38.418429 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 13:37:38 2022] *** May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: nodename: dv-cinder-843278-1-iscsi May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: machine: x86_64 May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: clock source: unix May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: pcre jit disabled May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: detected number of CPU cores: 4 May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: current working directory: / May 25 13:37:38.420714 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: detected binary path: /usr/bin/uwsgi-core May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: your processes number limit is 47680 May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: your memory page size is 4096 bytes May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: detected max file descriptor number: 2048 May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: lock engine: pthread robust mutexes May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: thunder lock: enabled May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 13:37:38.421359 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 13:37:38.466365 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: Python main interpreter initialized at 0x55bc509ae9c0 May 25 13:37:38.466365 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: python threads support enabled May 25 13:37:38.466365 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: your server socket listen backlog is limited to 100 connections May 25 13:37:38.466365 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: your mercy for graceful operations on workers is 90 seconds May 25 13:37:38.466676 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: mapped 403077 bytes (393 KB) for 2 cores May 25 13:37:38.466772 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: *** Operational MODE: preforking *** May 25 13:37:38.467039 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: *** uWSGI is running in multiple interpreter mode *** May 25 13:37:38.467039 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: spawned uWSGI master process (pid: 80164) May 25 13:37:38.467226 dv-cinder-843278-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 13:37:38.468294 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: spawned uWSGI worker 1 (pid: 80168, cores: 1) May 25 13:37:38.468294 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: spawned uWSGI worker 2 (pid: 80169, cores: 1) May 25 13:37:38.468294 dv-cinder-843278-1-iscsi devstack@keystone.service[80164]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 13:37:44.111739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:37:44.111739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes May 25 13:37:44.112558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:37:44.112558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes May 25 13:37:44.145781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:37:44.145781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 25 13:37:44.146296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:37:44.146296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 25 13:37:44.162243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 13:37:44.162440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 13:37:44.162631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 13:37:44.162899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 13:37:44.163077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 13:37:44.163328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.163573 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.163772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.163984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.164174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.164384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166623 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.166786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.167024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.167269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.167515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.167761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.167956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.168139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.168382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.168627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.168861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.169096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.169329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.169563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.169810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.170052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.170326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.170811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.170994 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.171224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.171451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.171628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.171871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.172131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.172364 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.172593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.172834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173009 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.173940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.174195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.174431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.174693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.174932 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.175181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.175420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.175668 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.175903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.176170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.176416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.176664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.176898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.177149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.177387 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.177634 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.177880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.178119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.178375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.178630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.178875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.179110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.179363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.179633 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.179886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.180158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.180394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.181398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.181635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.181872 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.182124 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.182374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.182607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.182838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.183085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.183333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.183574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.183823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.184076 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.184314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.184555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.184785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.185015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.185246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.185475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.185712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.185959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.186189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.186427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.186657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.186892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.187130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.187376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.187606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.187845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.188106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.188346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.188585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.188825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.189058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.189307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.189544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.189781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.190013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.190269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.190506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.190736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.190974 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.191208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.191475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.191711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.191977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.192224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.192471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.192709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.192945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.193189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.193429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.193667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.193910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.194149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.194238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 13:37:44.194391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.194469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 13:37:44.194599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] command line args: [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 13:37:44.194681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.194849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.194934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 13:37:44.195039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ================================================================================ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 13:37:44.195116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.195380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.195551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_dir = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.195637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.195741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.195818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.195953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_source = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.196052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.196225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] debug = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.196322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.196471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.196574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.196714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.196794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.196897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.196983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.197158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.197236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.197401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.197478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.197625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.197700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.197799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_config_append = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.197882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.197978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.198070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.198151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_dir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.198293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.198372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.198529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.198614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.198776 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.198866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.199014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.199092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.199269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.199347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.199432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.199527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.199606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.199766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.199850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.200020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.200115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.200252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.200330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.200547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.200626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.200778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.200864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.201007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.201085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.201236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.201314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.201474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.201560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.201708 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.201786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.201951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.202182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.202267 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.202413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] publish_errors = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.202491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.202646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.202768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.202868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.203003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.203110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.203206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.203309 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.203413 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.203519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.203622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.203720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] standard_threads = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.203805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.203980 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.204077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.204193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.204273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.204398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.204502 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.204587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_journal = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.204701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_json = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.204833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.204911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_stderr = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.205038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_syslog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.205229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:37:44.205461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205591 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.205973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206308 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.206939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207254 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207501 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.207892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.208984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209399 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209657 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.209961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210467 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210643 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.210983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211412 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.211966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212828 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.212982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213777 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.213947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214517 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.214988 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215687 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.215914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216808 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.216886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.217985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218764 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.218993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.219977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220439 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.220898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.221999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222242 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.222959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223191 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223492 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.223903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224711 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.224953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225567 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.225887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226053 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226764 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.226843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.227004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.227843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228044 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228286 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.228945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.229978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.230918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.external = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.231878 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.password = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232672 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.token = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.232910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.233865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234587 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.234819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.235980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236788 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.236956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237765 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.237925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238450 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238567 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.238934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239552 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239665 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.239914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.240933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.241989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.242889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.243585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.243670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.243731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.243915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244192 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244449 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.244909 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.245887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.246846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.247988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.248953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.249985 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250233 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.250933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251087 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.251874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 13:37:44.252782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.252988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.253237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.253485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.253722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.253959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.254197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.254428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.254670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.254932 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.255165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.255414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.255662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.255911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.256171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.256403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.256641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.256878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.257118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.257449 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.257704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.257955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.258206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.258457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.258732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.258966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.259218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.259459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.259702 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.259961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.260206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.260455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.260705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.260945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.261202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.261435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.261682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.261922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.262160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.262408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.262656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.262916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.263182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.263424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.263668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.263937 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.264191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.264430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.264683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.264935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.265232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.265522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.265802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.266063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.266303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.266555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.266790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.267043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.267291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.267541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.267792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.268067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.268318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.268568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.268817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.269051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.269299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.269548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.269798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.270045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.270443 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.270682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.270934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.271185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.271434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.271684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.271953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.272206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.272470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.272715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.272951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.273186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.273419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.273673 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.273928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.274691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.275029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.275275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.275512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.275753 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.276019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.276274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.276514 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.276774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.277023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.277261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.277511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.277756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.278007 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.278245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.278486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.278722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.278968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.279218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.279450 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.279694 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.280017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.280284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.280535 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.280785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.281029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.281273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.281512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.281754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.281995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.282251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.282504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.282744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.282993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.283231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.283481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.283720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.283853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.284009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.284140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.284283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.284412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.284555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.284683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:37:44.284811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 13:37:44.292871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.310172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.310405 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.310635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.320490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.321940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.322169 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.339699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.339948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.375640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.377928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.426607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.426840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.427636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.427874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.492565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.492792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.493650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494382 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.494788 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.517779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518553 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.518989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.519120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.519211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.519332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.530395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.530862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.542816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.543902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.544005 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.544147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.544230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.544384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.544603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:37:44.562233 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.562445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:37:44.562530 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.562692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:37:44.562774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.562971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:37:44.563052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.563211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:37:44.563297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.563479 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.563561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.563730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.563860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.564006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.564102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.564265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.575561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.575836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.576126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.576405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.577608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.577885 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.578151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.578431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.584507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.586571 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.588329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.588606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.588886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.589168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.589449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.589707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.589972 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.590352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.590435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.590622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.590716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.590895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.590977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.591160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.591241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.591466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.591838 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.592136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.592413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.592686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.592960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.593229 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.617724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.620398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.620672 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.620782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.622507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.622782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.625983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.626215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.626495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.628119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.628400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.628678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.634461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.634733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.636654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.636937 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.639215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.639616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.639899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.640204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.640477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.640751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.641019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.641286 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.641447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.641844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.642129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.642415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.642695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.642974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.643246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.643513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.660115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.660391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.662346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.662624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.665598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.667848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.669264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.669557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.671517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.671813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.674756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.676968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.678383 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.681151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.682852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.683148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.683426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.683713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.684017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.684312 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.684961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.685259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.685551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.685847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.686130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.686408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.703196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.703477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.703739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.704026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.705502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.705787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.706053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.706316 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.712022 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.713798 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.714072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.714441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.716162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.716451 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.717145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.717422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.717699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.717988 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.718263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.719513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.719790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.720102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.720386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.720662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.732732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.733010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.733297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.733574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.733848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.734117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.734392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.734874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.735155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.735436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.735722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.736033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.736324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.736599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.753170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.755121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.756438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.758437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.759892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.761510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.761962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.763147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.763431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.763579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.763719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.765226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.765513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.765806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.771120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.772745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.773243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.775077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.776026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.776323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.776609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.776879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.778434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.778763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.779101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.779430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.785955 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.786240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.786616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.788538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.788824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.789203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.795074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.795349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.795627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.795908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.796202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.796477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.796758 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.797023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.797297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.797561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.797673 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:37:44.797826 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.797952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798233 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.798760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.799028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.799305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.799586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.799853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.800156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.800437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.800698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.800971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.801238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:37:44.837803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bc509ae9c0 pid: 80169 (default app) May 25 13:37:44.840534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bc509ae9c0 pid: 80168 (default app) May 25 13:37:44.841035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15194-38e3-41d2-be78-430e20265e53 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.841329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15194-38e3-41d2-be78-430e20265e53 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.841607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd15194-38e3-41d2-be78-430e20265e53 None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.845967 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:52.980372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.980929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.980929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.986631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:37:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:52.991766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.992221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.992353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.065171 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] Registered counter for database keystone May 25 13:37:53.095177 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 13:37:53.105768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 13:37:53.298507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 13:37:53.298844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 13:37:53.355323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:53.356980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:53.492942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:37:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 13:37:53.680963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.749864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: INFO dbcounter [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] Registered counter for database keystone May 25 13:37:53.769844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG oslo_db.sqlalchemy.engines [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 13:37:53.776695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writer thread running {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 13:37:53.895437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a98715-b926-4c39-a7cc-cc8e90d509d1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.900528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.901199 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.901489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.901773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.909953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:53.960423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 13:37:53.960715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9753a-5fc7-4edd-a1fb-2b03954661ae admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:53.970863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 13:37:53.977232 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.991389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72c45a-f167-46bf-9e96-21f290e4e7b4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.995814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.996966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.997255 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.997537 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] PATH_INFO: `/v3/projects/c319a094752b41428929e5f2edeaa06e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.999021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] RBAC: Authorizing `identity:get_project(project_id=c319a094752b41428929e5f2edeaa06e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.161701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 13:37:54.162042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a189490-6447-4a03-9996-a34a6d316f26 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:54.167014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/projects/c319a094752b41428929e5f2edeaa06e => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:56.383751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a062-a239-4873-85c5-ca688a54b03a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.384253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a062-a239-4873-85c5-ca688a54b03a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.384360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a062-a239-4873-85c5-ca688a54b03a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.388524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:37:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:56.394082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.394499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.394646 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.413739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:56.415935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:56.423811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:37:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 13:37:56.445857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a062-a239-4873-85c5-ca688a54b03a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:56.457682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a062-a239-4873-85c5-ca688a54b03a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.462790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.463423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.463711 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.464090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.471701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.473507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11420353-5dd4-4b03-9e12-80e47fb0e2c4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.482586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Wed May 25 13:37:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 13:37:56.487935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:56.497977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd890e-2bb2-4e7f-97e7-54cd8d3a283e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.502727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.503591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.503881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.504195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.505595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.507444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8788b-e04b-4c0f-aeb7-033b6922a930 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.509828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Wed May 25 13:37:56 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:58.777860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d366a-212e-4b1b-b1e2-b2d1e14201c6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:58.778394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d366a-212e-4b1b-b1e2-b2d1e14201c6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:58.778479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d366a-212e-4b1b-b1e2-b2d1e14201c6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:58.779765 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:58.785026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:58.785462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:58.785582 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:58.801940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:58.803112 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:58.845407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89bbd9-c3c8-490e-aa05-08af06bd8de4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:58.845697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89bbd9-c3c8-490e-aa05-08af06bd8de4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:58.845978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89bbd9-c3c8-490e-aa05-08af06bd8de4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:58.847317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:58.852755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:58.853044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:58.853739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:58.884127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:58.893940 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 13:37:58.894353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 13:37:58.906771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:58.931989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:58.941598 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:58.943210 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:59.014430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77baefcf-b1f1-40a4-9f1b-21262256c388 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.042459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:59.047618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.048447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.048745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.049029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.060648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:59.063650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:59.072648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:59.073796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:37:59.074592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:37:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:59.079712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.162662 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.168737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96215d1e-69dc-4629-a708-90b139a2a7da None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.171036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.182005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6393a-6185-44ae-b100-b0a36f0271bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.207933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.208845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.209132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.209407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.217469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.218173 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.218462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.218741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.219495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:59.219809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.221277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.233778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:59.239369 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.276842 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.278364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.335743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d6624-e796-446a-9398-19b4a4c425d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.373445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.374311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.374620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.374891 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.375444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:37:59.376127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.377595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.379202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:59.381073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.384029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.384511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.400456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.479262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8caa-06fe-4a8e-be2f-df3b767d96da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.485070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a59e3-f409-45c5-a7d4-2ecaafa6726b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.512620 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.513408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.513695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.513976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.517766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.518554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.518843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.519131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.520349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.521812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.523934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/18] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:59.529553 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.533116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.534643 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:37:59.556693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:59.557520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:59.558293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:37:59.558293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:37:59.558293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:37:59.558293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:37:59.558293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 9/19] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:59.564874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.610830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.623429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.631858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ff986-c2b1-4750-94b9-9e785bd0c183 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.667525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.668220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.668321 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b96c-51f0-42b9-bc2c-6ba5564c51c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.668512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.668796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.669732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.domain.id=default, user.id=47296119d0ef495489c4b5806001b954) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:59.669961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.671418 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ab91d-1c6e-47b0-80dd-39db5a6eedfd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.678266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1322 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/role_assignments?user.id=47296119d0ef495489c4b5806001b954&scope.domain.id=default&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:59.818745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.819419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.819703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.820003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.821297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:59.821600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.823032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89626a-dce6-4b5f-815d-b6b78c97b660 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.842541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:02.185005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2668370-7a62-433d-9a90-da7fbedab2cb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.185536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2668370-7a62-433d-9a90-da7fbedab2cb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.185619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2668370-7a62-433d-9a90-da7fbedab2cb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.187210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:02.206027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e1d2-55c8-42e5-96cf-d513888001bc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.206495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e1d2-55c8-42e5-96cf-d513888001bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.206591 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21e1d2-55c8-42e5-96cf-d513888001bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.232198 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ec21e1d2-55c8-42e5-96cf-d513888001bc None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:02.233432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ec21e1d2-55c8-42e5-96cf-d513888001bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:02.316884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 11/23] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:02.321438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ddae-75e4-4165-8ed4-fc97556f0655 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.321726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ddae-75e4-4165-8ed4-fc97556f0655 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.322008 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77ddae-75e4-4165-8ed4-fc97556f0655 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.323608 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/24] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:02.337869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.338161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.338442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.361192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77ddae-75e4-4165-8ed4-fc97556f0655 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:02.389472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:02.390691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:02.522430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:02.554177 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77ddae-75e4-4165-8ed4-fc97556f0655 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:02.578872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 12/25] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:02.596204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:02.597274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.597729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.598198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.618142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:02.623117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:02.632403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:02.633469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:02.634213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/26] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:02.638441 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:02.649607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:02.746308 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a054d5-6c0e-4180-a9e1-2546f85ea7e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:02.785852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:02.786883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.787359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.787829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.788814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:02.790412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:02.792045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07609a11-a04e-4c28-b1ec-6552395b0d16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:02.800790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/27] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:02.802332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e05f1-5cac-474b-8246-c64b5eb9c4fe None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.802819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e05f1-5cac-474b-8246-c64b5eb9c4fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.803304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e05f1-5cac-474b-8246-c64b5eb9c4fe None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.805149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/28] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:02.812783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e05f1-5cac-474b-8246-c64b5eb9c4fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:02.830627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e5e70-f5ac-4211-93c2-325116b947ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:02.868979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:02.870304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.870811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.871279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.884146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:02.886231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7afb790a-a35f-4341-b745-fec644280d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:02.906416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:02.907355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:02.908162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:02.908162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:02.908162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:02.908162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:02.910103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 13/29] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:02.911647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.912147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.912644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.932435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:02.933910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:02.955024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:02.964952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:03.032673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e05f1-5cac-474b-8246-c64b5eb9c4fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.042555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 14/30] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:03.043738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.109836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.111497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.111789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.112094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.118556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.119122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.121402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/31] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:02 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:03.122055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.252529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6165819-c825-43bb-89bb-f97ddc84f2e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.278822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec384e9-7135-4353-954c-a846e0075e22 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.292055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.293132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.293651 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.294129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.295132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.296752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.299604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:03.344658 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.347862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.347862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.347862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.347862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:03.347862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.348354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.372837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 15/32] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:03.373515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.400442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:03.457434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/33] 10.222.0.45 () {60 vars in 1131 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/roles => generated 219 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:03.465588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.499691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f302a-4606-40be-bd42-a3f0c486f49b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.589372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99161-0c90-492c-9ae9-950ed2237d03 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.593401 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.594071 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.594359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.594636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.596174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.597847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723bcb-91dc-4d17-97fa-fd79407c83bb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.599871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 16/34] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:03.604166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa751a-2176-47da-87a6-d57d37e2daed None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.604166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa751a-2176-47da-87a6-d57d37e2daed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.604166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa751a-2176-47da-87a6-d57d37e2daed None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.605167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 17/35] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:03.613661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.613845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.614127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.650004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.651146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.651691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.653066 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:03.653650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.654794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.657527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.658823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:03.740407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/36] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/projects => generated 301 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:03.741344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.765934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 18/37] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:03.806128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:03.815100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:03.855007 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:03.890298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf982-8805-4c5f-a90b-abe373a844f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.941162 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:03.942563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.942854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.943417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] PATH_INFO: `/v3/domains/default/users/47296119d0ef495489c4b5806001b954/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.946835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=47296119d0ef495489c4b5806001b954, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:03.948676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8409f9b-7649-48a9-9f11-c7611ae410a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:03.951822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa6d16-32fb-4a79-9bbc-331b4d4e0627 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:03.974639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/38] 10.222.0.45 () {58 vars in 1361 bytes} [Wed May 25 13:38:03 2022] PUT /identity/v3/domains/default/users/47296119d0ef495489c4b5806001b954/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:03.977150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2338-3cd4-499d-ba3d-76decb83ff10 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.977959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2338-3cd4-499d-ba3d-76decb83ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.978251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b2338-3cd4-499d-ba3d-76decb83ff10 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.979825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/39] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:03.986998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.987292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.987589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.003627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.004381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.004730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.005011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.005876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.007444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352d7b6-0e9b-4472-b790-cc777fde4793 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.012953 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:04.015299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:04.111860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/40] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:04.139848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.158887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 19/41] 10.222.0.45 () {60 vars in 1131 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/roles => generated 225 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:04.197839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739ad8-0902-4d13-bb0f-808b715d420d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.198304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739ad8-0902-4d13-bb0f-808b715d420d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.198708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739ad8-0902-4d13-bb0f-808b715d420d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.200018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.200725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 20/42] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:04.207639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.207963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.208232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.229201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:04.230921 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:04.288725 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98300f4e-8bf6-46c6-8905-9d6dc53a906e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.330779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.331460 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.331751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.332111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.332947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.334446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.336419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 21/43] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:04.354806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/44] 10.222.0.45 () {60 vars in 1131 bytes} [Wed May 25 13:38:04 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:38:04.360787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.378492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.507091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbacf52-b193-4e4f-9b33-a09393c2650e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.548122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.548122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.548122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.548122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.549374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.550968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d332e-3fdb-4639-9900-873f4c0e208e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.552860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 22/45] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:04.558347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.667770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.674778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6caf-2af2-4236-afed-6639c8dff806 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.708882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.709546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.709837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.710119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.711496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.713021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e66c7-9d59-4475-b63d-8f9b2c5da824 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.740206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/46] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:38:04 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:06.285223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975079e6-8294-4417-a435-56f0c2b60e93 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.285761 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975079e6-8294-4417-a435-56f0c2b60e93 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.285860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975079e6-8294-4417-a435-56f0c2b60e93 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.287170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 23/47] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:06.293454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.293848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.294147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.311129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:06.312356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:06.379315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/48] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:06.402769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975079e6-8294-4417-a435-56f0c2b60e93 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.478410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.505613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975079e6-8294-4417-a435-56f0c2b60e93 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.539455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.540339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.540635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.540918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.552053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:06.561622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:06.562476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:06.563223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 24/49] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:06.568595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.644810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.667538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6379a9-6b49-49d0-a443-35743fcdd441 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.700936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.701605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.701895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.702175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.702928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:06.703240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.704683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.712854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/50] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:06.718329 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.822020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaec2c-9eaf-4df2-bf3a-d95953e37041 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.858028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.858905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.859193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.859479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.860936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.862394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca5d8f-79d4-4b4d-afff-e10d76a76d61 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.864048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 25/51] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:06.869434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.967840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d184a0-91c1-4ee9-82d8-84ccecf0bfba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.974812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f270-79dc-44df-bb14-37935a4f083b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.975161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f270-79dc-44df-bb14-37935a4f083b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.975431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7f270-79dc-44df-bb14-37935a4f083b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.977934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 26/52] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:06.982864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.983159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.983448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.988902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.001757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:07.002951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:07.003082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.003761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.004089 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.004376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.005565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.007032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.008871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/53] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:07.014397 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.025844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.070826 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 27/54] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:07.096970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.118317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f6105-cc4f-4e97-8afd-090096ed338f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.152076 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.152736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.153028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.153311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.154150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.domain.id=default, user.id=47296119d0ef495489c4b5806001b954) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:07.154468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.155970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.161809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/55] 10.222.0.45 () {56 vars in 1322 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/role_assignments?user.id=47296119d0ef495489c4b5806001b954&scope.domain.id=default&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:07.199096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d4a85-56bd-47eb-8739-0b18bb00d47c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.231697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.237776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.238616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.238910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.239192 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.240493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.242001 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeade5ad-3d5d-4fa8-9f05-ba9554db2238 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.243677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 28/56] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:07.249432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.323674 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.359561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e87ea9-cb23-49df-932b-c282809bb6e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.394928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.395600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.395887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.396214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.396933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.398523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab608-91e6-4dd1-b2f4-adf3d6da664e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.424894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/57] 10.222.0.45 () {60 vars in 1140 bytes} [Wed May 25 13:38:07 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:09.630505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92944-0ab4-4509-baa2-7eca44bd42d8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.631007 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92944-0ab4-4509-baa2-7eca44bd42d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.631092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe92944-0ab4-4509-baa2-7eca44bd42d8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.633050 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 29/58] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:09.638825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.639236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.639395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.657132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:09.658337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:09.722877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/59] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:09.746959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92944-0ab4-4509-baa2-7eca44bd42d8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.846453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe92944-0ab4-4509-baa2-7eca44bd42d8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.879622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.880433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.880774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.881003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.882182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.883804 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0ce7b-0481-485f-8bac-838743e19cf7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.885415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 30/60] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:09.890709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.944479 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:09.988943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2b1c-b99f-481c-8998-bd69a440affd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:10.022900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:10.024088 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:10.024317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:10.024596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:10.025303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:10.026738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a97b-3a6a-4a23-acea-6da5d753c336 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:10.051497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/61] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:09 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:13.353846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2d24-c065-462f-ad22-3bc98fb8e0e6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.354293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2d24-c065-462f-ad22-3bc98fb8e0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.354409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2d24-c065-462f-ad22-3bc98fb8e0e6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.355979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 31/62] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:13.363505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.364374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.364851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.369268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d42e-662c-44e0-858b-7f05183ceb03 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.369568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d42e-662c-44e0-858b-7f05183ceb03 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.369857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d42e-662c-44e0-858b-7f05183ceb03 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.371355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 32/63] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:13.382148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.383308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.383806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.386602 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:13.387797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:13.409262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:13.410475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:13.516117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/64] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:13.530885 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.541202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 33/65] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:13.565965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.607857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.668339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.704642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0f4b-ff82-495d-b7b9-fa04bcf0b259 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.742115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.743224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.743661 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.744132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.755242 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:13.764351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.765224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:13.765961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:13.768333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/66] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.777106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.816959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f40984-9a4f-497c-9860-b8f6c87d8978 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.862081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.862807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.863094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.863375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.876028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c685dd59-11be-4a88-be0c-c5fc50277ad4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:13.884907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.885764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:13.886653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 34/67] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.887495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96bfa6-8995-4c15-a21e-71fcad99f1ab None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.887702 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96bfa6-8995-4c15-a21e-71fcad99f1ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.888023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96bfa6-8995-4c15-a21e-71fcad99f1ab None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.889403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 35/68] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:13.894074 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96bfa6-8995-4c15-a21e-71fcad99f1ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.908975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bba11-ff37-42fa-b939-94497b4b6d24 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.912246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.952203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.953522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.953995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.954420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.955354 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.956412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.958391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cf9dc-5712-41f1-b979-4f1bead8f919 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.962774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.971835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/69] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:13.973809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.974270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.975198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.998627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:14.000465 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:14.016748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96bfa6-8995-4c15-a21e-71fcad99f1ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.054680 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.055564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.055945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.056307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.057146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.057531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.059107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.076274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 36/70] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.077366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.087693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/71] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:14.089388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.194046 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ca8fd-93f1-45aa-8c9b-31b3e414a1e1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.198639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.207700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7491-0c9d-480c-95b8-8503ac512ccb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.216776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.232523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.233396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.233683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.233966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.235237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.236723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.238364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 37/72] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.239133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.243391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.244328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.244619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.244899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.264842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.266334 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228881d-3508-4ab5-8131-eddb906cefb2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.268120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/73] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.269365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d26f7-7814-4eaa-ae6c-e72749c57c86 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.269657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d26f7-7814-4eaa-ae6c-e72749c57c86 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.269938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d26f7-7814-4eaa-ae6c-e72749c57c86 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.271181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/74] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:14.272273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587413b7-909a-431c-9444-46c09aa35b38 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.272607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587413b7-909a-431c-9444-46c09aa35b38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.272859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587413b7-909a-431c-9444-46c09aa35b38 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.274656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/75] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:14.275374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587413b7-909a-431c-9444-46c09aa35b38 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.344410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7639d-0ad2-4db3-80f2-caef722a3e12 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.380413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.380987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587413b7-909a-431c-9444-46c09aa35b38 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.381329 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.381506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.381784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.392142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:14.401631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:14.402463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:14.403207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 38/76] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:14.404901 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.415266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.415966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.416260 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.416543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.435402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.453694 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.486622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.488134 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886450c6-1ff6-42be-b89a-c4dc8dde7e05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.504959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcecb5-3d94-4765-986b-51ba4fc1d49d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.540315 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.540962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.541252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.541530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.587858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/77] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:14.589180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149acf-48af-4ab9-b31f-a22f2dbb38b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.589484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149acf-48af-4ab9-b31f-a22f2dbb38b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.589764 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149acf-48af-4ab9-b31f-a22f2dbb38b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.605804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-de149acf-48af-4ab9-b31f-a22f2dbb38b1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:14.606996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de149acf-48af-4ab9-b31f-a22f2dbb38b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:14.612712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.614183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.654750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.664217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.677241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/78] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:14.678527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.678826 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.679814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.703005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:14.704321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:14.718188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 39/79] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:14.719142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.777021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/80] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:14.777866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.823009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eda5f7-3e16-4ad3-bb71-9e299560a787 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.856464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.857132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.857419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.857703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.858465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.858777 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.860201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.868981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 40/81] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.869784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.880357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e37706-9312-4983-90bd-edf90e018b50 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.881556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.891756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.916680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.917546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.917840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.918117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.919298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.920805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.922520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/82] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.923316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.971493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21fab3-4801-419a-91b7-573fc5d5d733 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.004872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.005669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.005968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.006248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.016928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:15.021071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c32708-f269-4f9a-9ffc-66cd199254be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:15.025486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:15.026411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:15.027144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 41/83] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:15.028212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.055149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.056031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.056316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.056657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.057763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.059206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.060905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/84] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.061655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.105864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.109126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.139146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742499ad-f53f-4fb7-a7a8-c2d486c22d0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.161225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe03e-8d63-4137-91b8-e7b06131f003 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.172441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.173240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.173527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.173805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.174980 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.176459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.178052 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 42/85] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.178812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.194927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.195890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.196203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.196476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.197676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.199116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.200761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/86] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.201518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.284250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f60842-e94d-4314-a8dc-9f484b64ed56 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.298963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e0764-bdc6-4630-ab8a-534fbbeb8579 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.313564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.318697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.319359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.319686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.320033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.321042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=47296119d0ef495489c4b5806001b954) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.321363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.322810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.331127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 43/87] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/role_assignments?user.id=47296119d0ef495489c4b5806001b954&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.331945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.334522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.344956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.345665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.345960 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.346243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.347043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.347364 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.348896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.360164 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/88] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.361149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.453324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893319c0-506f-4352-b04f-2a4ebd8862c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.487799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f56a-22be-46fa-b89a-47def512f5bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.489775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.490483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.490778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.491062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.492519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.492850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.494341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.500687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 44/89] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.501380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.536749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.537811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.538282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.539246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.540195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.541713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.566458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/90] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:15 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:15.573100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.604621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.641801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.646131 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9af14-e445-4fa5-8749-a36b4ead5dd5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.698702 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.699858 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.700429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.700958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.783108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.784972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.785362 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b331b-da44-420a-b046-33329738fb6f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.824211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.824529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.824719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.824990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.826242 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.827791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.830210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/91] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.835800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.859390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.899259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 45/92] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:15.904537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.962676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.967642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c222c9d-3ae7-4499-8d22-911764685203 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.011179 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.012245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.012831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.013313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.019604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bffea-60a0-429c-975c-26daed6d411d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.060774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.061687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.061981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.062266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.063666 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.064972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.066922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 46/93] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:16.072787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.094379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.096347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.111036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.182120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7df77-fb56-44ec-a30e-8f6060574f0c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.207222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.212847 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/94] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:16.218151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.228549 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.229975 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.230492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.231348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.232770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.233311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.235695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.243865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 47/95] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=798530c22da24d42b101d2f46d4939bb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:16.346966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d078-b327-4626-b498-c01176c2dfd6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.390909 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.391806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.392184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.392468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] PATH_INFO: `/v3/projects/cc12d1e1517c4fbdbcefe437c3473767` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.393790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] RBAC: Authorizing `identity:get_project(project_id=cc12d1e1517c4fbdbcefe437c3473767)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.395353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a9823-0f23-443f-80bd-c5e80decd312 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.397528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/96] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/projects/cc12d1e1517c4fbdbcefe437c3473767 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:16.405159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.519014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.524075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8529b-cabe-4b5f-8a7a-82f142ae085b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.560979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.561646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.561935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.562217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.563056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=cc12d1e1517c4fbdbcefe437c3473767, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.563374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.564849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4af46d-b294-4614-9d57-3569e0161ff1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.570965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 48/97] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=cc12d1e1517c4fbdbcefe437c3473767&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:18.045616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2452e-00b6-4c06-9737-1852283d1dcf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.046147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2452e-00b6-4c06-9737-1852283d1dcf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.046223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2452e-00b6-4c06-9737-1852283d1dcf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.047812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/98] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:18.052818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.053619 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.054749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.075760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:18.077186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:18.170686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 49/99] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:18.204924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2452e-00b6-4c06-9737-1852283d1dcf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.290881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.339741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2452e-00b6-4c06-9737-1852283d1dcf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.377335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.378141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.378431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.378720 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.390613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2cf0a663-1d7b-47e5-b32a-c6021ac47cd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:18.399267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:18.400201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:18.400956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/100] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:18.409525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.457544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8cacd-e015-480b-b2e3-c82ce8609d78 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.457865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8cacd-e015-480b-b2e3-c82ce8609d78 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.458110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8cacd-e015-480b-b2e3-c82ce8609d78 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.459631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/101] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:18.465198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.465491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.465777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.501139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:18.502370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:18.603272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.623520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/102] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:18.634067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b10a-6fdd-4e17-aa40-d437026b5f20 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.646917 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.716682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.717367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.717656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.717931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.718701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:18.719011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.720415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.720963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.746225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 50/103] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:18.751714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.758324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4329676-4fe4-44fe-bcff-370e0a00265b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.796838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.798001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.798293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.798576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.810989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:18.819461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:18.821041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:18.821802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/104] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:18.841695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.955300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fe537-b31f-4f4d-aa8c-7908cac3de6a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.016650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.017543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.017832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.018110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.019352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.020825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.022626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 51/105] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:19.027762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.082524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.108229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855bb13-2ad4-4e3b-b58d-b081cd9a08f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.161086 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.211058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.212060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.212514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.212939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.213860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:19.214317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.215881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.233049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/106] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:19.245037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.254034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa1243-571c-422b-a1b3-07631e86937f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.309896 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.311905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.316549 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.317061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.318600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.320616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.322742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 52/107] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:19.328386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.404828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bf23-c3f6-46a9-ae66-44069435bbb2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.470420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.471225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.471511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.471790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.473783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.475246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4174767-8a66-4647-b2ec-556a415a0c04 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.477843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/108] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:19.479051 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbea5c-5f09-42a7-9c44-3efed8976a27 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.479349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbea5c-5f09-42a7-9c44-3efed8976a27 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.479632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbea5c-5f09-42a7-9c44-3efed8976a27 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.480910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/109] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:19.489013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbea5c-5f09-42a7-9c44-3efed8976a27 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.517268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.535855 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78e269-d4ee-4572-a85c-74284a7d818b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.617260 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.618177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.618467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.618745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/users/47296119d0ef495489c4b5806001b954/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.618883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.621107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, user_id=47296119d0ef495489c4b5806001b954, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.622703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f1de-7053-414d-ba63-10d914d2c3a6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.645378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 53/110] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:19 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/users/47296119d0ef495489c4b5806001b954/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:19.655146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171829-cc51-420d-8715-304cbcba48ee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.655964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171829-cc51-420d-8715-304cbcba48ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.657097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39171829-cc51-420d-8715-304cbcba48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.697368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-39171829-cc51-420d-8715-304cbcba48ee None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:19.709722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-39171829-cc51-420d-8715-304cbcba48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:19.742655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbea5c-5f09-42a7-9c44-3efed8976a27 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.792543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.793764 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.794065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.794355 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.796060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.798485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7ce22-3219-435b-b75e-b4efb93b393d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.800489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/111] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:19.801904 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8bc1d-047c-43ae-81dc-85ef9c4c8a2c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.802207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8bc1d-047c-43ae-81dc-85ef9c4c8a2c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.802492 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8bc1d-047c-43ae-81dc-85ef9c4c8a2c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.802827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 54/112] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:19.810194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/113] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:19.810194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8bc1d-047c-43ae-81dc-85ef9c4c8a2c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.811567 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e21f5-9ea7-43e3-b2e5-2b61b8955a97 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.812120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e21f5-9ea7-43e3-b2e5-2b61b8955a97 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.812644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e21f5-9ea7-43e3-b2e5-2b61b8955a97 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.814512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 55/114] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:19.816182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b495b-ebed-4f0b-b4f2-91be0d2e26aa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.816795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b495b-ebed-4f0b-b4f2-91be0d2e26aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.817274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b495b-ebed-4f0b-b4f2-91be0d2e26aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.857080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-257b495b-ebed-4f0b-b4f2-91be0d2e26aa None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:19.858822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-257b495b-ebed-4f0b-b4f2-91be0d2e26aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:19.946505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8bc1d-047c-43ae-81dc-85ef9c4c8a2c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.953706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 56/115] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:19.956503 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.957002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.957449 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.980948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.986063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:19.987958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:19.992719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.000640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.001620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.001984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.002391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/users/11af095e02cd4d71b25cc4525e9fdf03/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.004767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, user_id=11af095e02cd4d71b25cc4525e9fdf03, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.006338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.028098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/116] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:19 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/users/11af095e02cd4d71b25cc4525e9fdf03/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:20.029172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.084189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 57/117] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:20.090046 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.173768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156da6a-808b-4ce0-9cbf-0603a00d6058 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.215279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.216122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.216411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.216691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.218203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.219767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb38c4-b079-46f7-856c-fee61ada21fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.221900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/118] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:20.223226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d0bb-53c2-4a01-b67c-60e5797da784 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.223522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d0bb-53c2-4a01-b67c-60e5797da784 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.223822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d0bb-53c2-4a01-b67c-60e5797da784 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.224537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b336df6-fbdd-4534-824b-53cda364f5de None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.230474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/119] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:20.231174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d0bb-53c2-4a01-b67c-60e5797da784 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.266227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.267069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.267355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.267728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.284657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.286183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.289000 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:20.300340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:20.301244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:20.302038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:20.306004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 58/120] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:20.307055 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.366310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d0bb-53c2-4a01-b67c-60e5797da784 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.412149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.412963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.413298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.413737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.428415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-79397528-2f7f-4096-af6a-877ad4214dab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:20.439855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:20.440756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:20.441519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/121] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:20.442880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.442880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.442880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.448526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c23eef-cc75-4338-981b-5c9738fbf0ca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.471079 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:20.472437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:20.488429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.489313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.489795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.490219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.491091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.493523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.526982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 59/122] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:38:20 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:20.528175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.564202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/123] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:20.570903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.616882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.617512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.736460 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486249ae-3d3c-4a40-85c7-50a3ea9522d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.758753 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dd6f5-4b5f-4d11-a20d-330a767ed0ab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.789855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.790462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.790752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.791034 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.791799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:20.792324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.793755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.804310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/124] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:20.805564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.815329 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.816056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.816350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.816957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.817390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:20.817708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.819110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.829304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 60/125] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:20.831958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.916209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.923388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b79f8-548b-4384-bf9e-83756ccba06b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.939688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cfcf-032a-48c2-b824-4aeb20e3f515 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.960897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.967839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.969126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.969629 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.970125 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.985833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:20.987713 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.988640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.988936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.989219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] PATH_INFO: `/v3/users/514acb994818468099c0b1935c905f80` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:20.995877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:20.996786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:20.997540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:21.000324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/126] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:21.001327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.009626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=514acb994818468099c0b1935c905f80)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.011127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.014196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 61/127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/users/514acb994818468099c0b1935c905f80 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.015062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.138886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ef3a-ebf4-46a7-83e3-c092e4309ad7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.178959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.179674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.179990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.180281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.183946 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:21.183946 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.183946 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.193149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 62/128] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.193904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.195940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b500e-3040-443a-a081-87a03760ece5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.244297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.291902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.293197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.293744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.294225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.295707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.297434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.299301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/129] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.300316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.306296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1d46c-df0d-42cf-ae79-60deed267c72 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.333590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.356601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.357272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.357618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.357906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.439114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.441045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.496292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.540732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d08544-ad54-4951-9eac-ff7f3fd762f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.552044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 63/130] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:21.552993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.653443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.654707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.655199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.655704 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.662787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9585d-8c75-4051-987d-02a5107790e7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.664932 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.666642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.668583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/131] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.669570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.703373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.704197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.704489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.704773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.709307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.709307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.709504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 64/132] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.710409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.724226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.828308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6009b0-62ee-4cf2-acd6-898df444ab0f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.858874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.859978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.860279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.860563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] PATH_INFO: `/v3/projects/cc12d1e1517c4fbdbcefe437c3473767` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.861833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] RBAC: Authorizing `identity:get_project(project_id=cc12d1e1517c4fbdbcefe437c3473767)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.863285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.865534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 65/133] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/projects/cc12d1e1517c4fbdbcefe437c3473767 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.866924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.884996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.917896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc5977-4d76-4c27-9492-7a91b00cf146 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.952325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.979111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190dc2e0-cc9e-49dc-ac32-ae8e24ce4d07 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.019816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.021421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.022051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.022340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/users/11af095e02cd4d71b25cc4525e9fdf03/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.025157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, user_id=11af095e02cd4d71b25cc4525e9fdf03, role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.026624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.037719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.038645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.038936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.039222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.041887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 66/134] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:21 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/users/11af095e02cd4d71b25cc4525e9fdf03/roles/798530c22da24d42b101d2f46d4939bb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:22.043113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.043650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.045143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.047127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/135] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:22.057298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.171257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e5b2a-c233-4d60-8895-a2a345cc2f0c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.208358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.220639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.222524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.223029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.223509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] PATH_INFO: `/v3/projects/cc12d1e1517c4fbdbcefe437c3473767/users/11af095e02cd4d71b25cc4525e9fdf03/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.230508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc12d1e1517c4fbdbcefe437c3473767, user_id=11af095e02cd4d71b25cc4525e9fdf03, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.232429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851d7b7-f515-462f-a8f1-802a01aca1e4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.260116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 67/136] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:22 2022] PUT /identity/v3/projects/cc12d1e1517c4fbdbcefe437c3473767/users/11af095e02cd4d71b25cc4525e9fdf03/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:22.324083 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c288cd-bb26-480d-8cf3-56f717290b85 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.390295 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.391265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.391708 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.392167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.393197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=514acb994818468099c0b1935c905f80) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:22.393734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.395492 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579cef9-d90b-4186-9960-eaab271f659c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.403174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/137] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/role_assignments?user.id=514acb994818468099c0b1935c905f80&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:22.522495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4079e-4b68-441d-bf58-81ff6d39bd69 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.523084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4079e-4b68-441d-bf58-81ff6d39bd69 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.523528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4079e-4b68-441d-bf58-81ff6d39bd69 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.525278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 68/138] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:22.531414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.531717 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.532033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.550610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:22.551887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:22.577534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.658427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/139] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:22.699321 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4079e-4b68-441d-bf58-81ff6d39bd69 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.828709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4079e-4b68-441d-bf58-81ff6d39bd69 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.871081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.871897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.872214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.872499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.889756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-405a1f09-30ca-49b0-87d2-552728f0af70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:22.898636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:22.899502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:22.900301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 69/140] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:22.907297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.919450 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38275fec-bd3a-4ebc-9c3c-08376696b67b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.919742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38275fec-bd3a-4ebc-9c3c-08376696b67b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.920053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38275fec-bd3a-4ebc-9c3c-08376696b67b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.921444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 70/141] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:22.927651 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.927964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.928251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.963959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:22.965182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:23.066333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.099879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a0790-d8e7-47c4-bcaa-cba407f0ef13 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.106701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 71/142] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:23.157067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.166999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.168143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.168668 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.169155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.170148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:23.170752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.172433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.183126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/143] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:23.189280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.214793 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.318914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3c828-4430-4f85-82cc-84c8427427e7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.353451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160de2-0e4d-4e29-a5a8-3505042e79f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.362221 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.363303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.363763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.364222 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.377110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:23.385733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:23.386577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:23.387312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:23.389707 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 72/144] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:23.396907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.415808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.416797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.417091 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.417374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] PATH_INFO: `/v3/users/47296119d0ef495489c4b5806001b954` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.418617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] RBAC: Authorizing `identity:get_user(user_id=47296119d0ef495489c4b5806001b954)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.420127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.421783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/145] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/users/47296119d0ef495489c4b5806001b954 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:23.427415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.513925 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.560321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.576540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab83531-71da-4ceb-8f5a-ccf12da71af8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.613536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.614122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.614414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.614698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.711431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.712946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a493-1a28-4d52-b0ff-acff5f04922a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.717587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2dc8c-3f97-4a80-92a5-c0fa0c20a4a7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.779213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.779891 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.780219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.780507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.781280 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:23.781596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.782996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58b86-b6ae-48fe-bdbf-d0d62c9c4899 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.800976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 73/146] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:23.803710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba6eb-9ddb-46cd-a858-389e63768884 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.804116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba6eb-9ddb-46cd-a858-389e63768884 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.804403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba6eb-9ddb-46cd-a858-389e63768884 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.806783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 74/147] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:23.813622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ba6eb-9ddb-46cd-a858-389e63768884 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.879272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.909126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/148] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:38:23.910747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.911236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.911721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.953069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:23.954651 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:23.994444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ba6eb-9ddb-46cd-a858-389e63768884 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.996103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.058141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.060231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.060524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.060965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.063444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.064940 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.066623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 75/149] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.068383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.086249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/150] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:24.087357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.274859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a6ab5-cee4-4c23-abf6-adbe35f7e5fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.279324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e941f64-8257-4d59-a573-59f65570373b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.304356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.345209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.345429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.346835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.347358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.347874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.350082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.350622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.351671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.354226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.360109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.365069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 76/151] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.365892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.462606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.464336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.482862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.573245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490cda8-e1df-4c3e-8199-3fdbae5e0dfc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.592387 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/152] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:24.593332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.613723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.614535 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.614820 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.615096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.618176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.619664 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.622560 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.623227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 77/153] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.624932 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.756352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def09f7-c7a4-449f-a653-51759982c698 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.765020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.793303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.794186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.794473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.794751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.796022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.797495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.799221 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 78/154] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.800096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.833943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d2e7-8a52-46cc-94d1-e61a2660d6c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.905294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178f7d-c76b-4d23-a4ef-918799a184d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.942932 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.943647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.943958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.944245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.945344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.946856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.967270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.967867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.968231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.968675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.969788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=47296119d0ef495489c4b5806001b954) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:24.970171 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.975407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262afe5-ed00-46cc-9a7f-ce0e762c0e71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.981668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 79/155] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:24 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:24.983133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.991639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/156] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/role_assignments?user.id=47296119d0ef495489c4b5806001b954&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.023236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.107589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a045d-dc40-411c-be9a-01ff568e97d8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.146119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.147086 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.148301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.148800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.149836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.150388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.152154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41854288-329f-45c3-84aa-2f4f58e3ef1b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.159582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 80/157] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.181117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00fe4c-ee18-4619-846f-882a2cadf78c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.181404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00fe4c-ee18-4619-846f-882a2cadf78c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.181677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00fe4c-ee18-4619-846f-882a2cadf78c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.183400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/158] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:25.189820 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.190104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.190376 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.216635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:25.217877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:25.273343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a3c0-423b-46f1-963d-609e5a9aa039 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.273743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a3c0-423b-46f1-963d-609e5a9aa039 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.274029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a3c0-423b-46f1-963d-609e5a9aa039 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.275512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/159] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:25.281334 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d22a-6524-4093-8a65-922ef58374e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.281626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d22a-6524-4093-8a65-922ef58374e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.281888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d22a-6524-4093-8a65-922ef58374e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.313066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db70d22a-6524-4093-8a65-922ef58374e7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:25.314350 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db70d22a-6524-4093-8a65-922ef58374e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:25.378607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 81/160] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:25.418774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.458318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.463781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/161] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:25.465189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9455-b66a-4688-a3a5-b2aa6c6dea7a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.465548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9455-b66a-4688-a3a5-b2aa6c6dea7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.465828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b9455-b66a-4688-a3a5-b2aa6c6dea7a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.467234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/162] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:25.472745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.473030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.473305 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.494336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:25.495650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:25.501155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.561083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51003d-7891-4833-9f4a-7bf87597f180 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.579200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/163] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:25.579445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.617135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.617916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.618242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.618546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.632617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:25.643484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:25.644610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:25.645520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 82/164] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:25.647005 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.702916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc3b5-b229-4eb0-9e73-ac0081b1868d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.742930 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.743882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.744295 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.744694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.758069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.763102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:25.763245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:25.774931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:25.775838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:25.776753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/165] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:25.777624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.784870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c70550-d00a-4686-9f35-16fdcd38269d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.826403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.827208 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.827496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.827771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.839785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:25.848447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:25.849375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:25.850152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 83/166] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:25.850904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.899155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790028ff-e33b-4e50-88fa-a2f6cd0f5095 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.939167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.939848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.940163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.940442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.941790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.942106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.944021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.955544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/167] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.961443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.975204 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d42c-4338-42eb-a103-e4269513655b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.035884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.036657 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.036938 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.037212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.038822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:26.040168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.041825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.043460 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.056081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 84/168] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.057238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.071671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.098474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56404c68-91f8-45be-80ed-dc40b0913c3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.138391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.139061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.139344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.139619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.140408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:26.140713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.142113 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.152159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/169] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.152914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.175334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2c424-9b00-4dc4-9ef1-e02dd4b89f26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.229710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.231006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.231504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.232002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.233457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.235164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.237213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 85/170] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.242784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.272753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb1681-c776-4cd2-9b07-5cb26d2d08e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.297913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.311942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.312833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.313114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.313384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] PATH_INFO: `/v3/users/514acb994818468099c0b1935c905f80` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.314590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] RBAC: Authorizing `identity:get_user(user_id=514acb994818468099c0b1935c905f80)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.316078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.317919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/171] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/users/514acb994818468099c0b1935c905f80 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.318462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.436124 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945061-c139-4b3b-847a-73e4f20ae05a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.442222 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.472568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.473213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.473490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.473763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.492817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f5f36-9030-4001-afe7-c77b32cc217b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.540078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.549258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.549402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.550490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.550809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.551207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.559946 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] PATH_INFO: `/v3/users/11af095e02cd4d71b25cc4525e9fdf03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.561578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] RBAC: Authorizing `identity:get_user(user_id=11af095e02cd4d71b25cc4525e9fdf03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.563226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.565143 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 86/172] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/users/11af095e02cd4d71b25cc4525e9fdf03 => generated 275 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.566200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.663147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/173] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:26.664401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.762384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.771997 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8a299-7229-4a5e-803d-c0a4e822757a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.793537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb16755-08ae-4860-82c6-0b628ae488d4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.817635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.818283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.818565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.818837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.842391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.843314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.843700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.844012 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.845504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.847120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.849306 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 87/174] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.850086 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.872311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.905190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.906665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.999417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.010905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/175] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:27.016302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.033286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eacbab7-9168-45ed-a1c8-65af9afdc3a9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.103952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.105395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.105887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.106369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.107981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.109676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.111591 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 88/176] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.113756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.132508 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b899-0c23-4d18-b9f7-709f605018e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.169966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.170855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.171130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.171400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/users/514acb994818468099c0b1935c905f80/roles/efeb4abc22fe4ff19bcabe767842fc2d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.175068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, user_id=514acb994818468099c0b1935c905f80, role_id=efeb4abc22fe4ff19bcabe767842fc2d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.176612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.201371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/177] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:27 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/users/514acb994818468099c0b1935c905f80/roles/efeb4abc22fe4ff19bcabe767842fc2d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:27.202206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.282792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.312321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.314134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52772d-58bd-4773-9d36-fba841b1c0b9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.384379 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.385512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.385937 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.386376 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] PATH_INFO: `/v3/projects/cc12d1e1517c4fbdbcefe437c3473767` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.387757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] RBAC: Authorizing `identity:get_project(project_id=cc12d1e1517c4fbdbcefe437c3473767)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.389407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.396368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 89/178] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/projects/cc12d1e1517c4fbdbcefe437c3473767 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.398104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.419951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240748-f217-47ee-a862-a5686db48d30 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.497313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.498159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.498520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.498869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.499805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.500217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.501838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccf4ca-a31f-45b8-86c6-c87b1a15f898 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.518125 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/179] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=798530c22da24d42b101d2f46d4939bb => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.571658 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04b7c2-f84c-41a5-8249-c142e8125b26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.610631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.611299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.611589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.611870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.612752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=cc12d1e1517c4fbdbcefe437c3473767, user.id=11af095e02cd4d71b25cc4525e9fdf03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.613070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.614608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964b62-ae38-449b-8f4b-f9830a37bd47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.621718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 90/180] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/role_assignments?user.id=11af095e02cd4d71b25cc4525e9fdf03&scope.project.id=cc12d1e1517c4fbdbcefe437c3473767&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.963308 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a6b2c-1358-4bc2-936a-82956ba53dac None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.963710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a6b2c-1358-4bc2-936a-82956ba53dac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.963885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a6b2c-1358-4bc2-936a-82956ba53dac None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.965420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/181] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:27.970930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.971472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.971972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.995806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:27.997348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:28.005091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.084444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 91/182] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:28.110400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a6b2c-1358-4bc2-936a-82956ba53dac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.231992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a6b2c-1358-4bc2-936a-82956ba53dac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.248254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.271285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.272160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.272606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.272944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.287003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:28.297844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:28.298740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:28.299513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/183] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:28.305662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.420417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6feb80-9949-4753-b1a2-e21ee9cb6733 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.456969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.457712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.458005 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.458283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.459141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:28.459455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.461156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.470255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 92/184] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:28.475307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.586794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3cab-823c-4b4e-94f8-b04f57b39531 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.623323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.624252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.624534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.624808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.639027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.640545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1846e7-e82b-4e66-9d05-304fa6beb5e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.642480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/185] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:28.648695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.651500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.749453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d228-387a-43f9-b5a2-b4811d95940f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.749788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d228-387a-43f9-b5a2-b4811d95940f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.750016 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d228-387a-43f9-b5a2-b4811d95940f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.751618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/186] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:28.757824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134cb14-ec24-4d18-8bba-f9b7d45f0ef1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.757824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134cb14-ec24-4d18-8bba-f9b7d45f0ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.758674 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134cb14-ec24-4d18-8bba-f9b7d45f0ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.762837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83aca7-374c-4b63-96ff-3fae3bc997ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.775780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2134cb14-ec24-4d18-8bba-f9b7d45f0ef1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:28.776990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2134cb14-ec24-4d18-8bba-f9b7d45f0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:28.798335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.798988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.799278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.799557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.804669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.849137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/187] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:28.854124 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9f68-25cb-4f50-8672-5fb774620d12 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.854499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9f68-25cb-4f50-8672-5fb774620d12 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.854850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d9f68-25cb-4f50-8672-5fb774620d12 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.856482 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/188] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:28.861911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.862207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.862549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.879828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:28.880374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.881062 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:28.881875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.883904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.957636 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/189] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:28.958831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.001641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 93/190] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:29.001641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.064775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.075833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce719a06-09e9-4ce5-bff9-61159e734199 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.105263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ebda-2a96-48cb-a839-0849a69f03de None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.107047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.113169 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.113982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.114390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.114669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.127420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:29.136174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:29.137220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/191] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:29.142734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.151211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.152263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.152818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.153239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.154616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.156406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.158221 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 94/192] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.159124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.243532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb5b4-a676-4ce1-887d-b471513e19af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.267174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e7250-c8f8-4360-9504-51677b4f29c7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.279121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.280014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.280298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.280572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.281757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.283215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.284925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/193] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.285669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.309416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.310517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.311468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.312060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.312505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.313396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:29.313846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.315361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.324639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 95/194] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.329874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.363305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.401165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3b0a4-f898-4d6f-b1a2-6b7a3964a788 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.435851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.436563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.436854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.437123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.438003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.438982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1c1d-b01c-4e72-9845-9fe2cc5c9ea3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.439459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.459357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/195] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:29 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:29.460574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.498200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.498884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.499183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.499448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.500723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=16ad9b32ecc740bcb9ef63fad18c96c2) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:29.501684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.503599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.513688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 96/196] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/role_assignments?user.id=16ad9b32ecc740bcb9ef63fad18c96c2&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.573986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.606249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e403f9-e12f-41d1-a74b-8ff72c006778 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.654326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.655153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.655447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.655729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.657096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.658614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3520b-e5e5-447a-aa23-3bb13d18fe5d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.660740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.667502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.786115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.797506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfdb44-43d7-4772-a5d9-9464bd997037 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.838072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.839360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.839843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.840455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.891675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02cb9-fbaa-480c-b7b4-3ca95814e186 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.896025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02cb9-fbaa-480c-b7b4-3ca95814e186 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.896321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02cb9-fbaa-480c-b7b4-3ca95814e186 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.897771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/198] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:29.904219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.904563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.904792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.923500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.924997 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.931329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:29.932849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:30.039138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/199] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:30.045243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.056199 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 97/200] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:30.061718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.083905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.164459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.205792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f3e7c-c2e7-408b-91d5-ea728e24edf9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.271216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcc83b-465f-4f94-93ca-8eb11872e5ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.274436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.275193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.275474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.275746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.277665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.279139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.285871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/201] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.286706 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.324447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.332057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.332057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.332057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.368063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:30.383575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:30.384801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:30.385735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:30.391882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 98/202] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:30.399147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.503581 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2754f-ffc0-4256-aa14-32d5472e9abd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.504083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.570815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.572522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.573558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.574127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.576322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=16ad9b32ecc740bcb9ef63fad18c96c2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:30.577594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.579802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.587305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95cf32-4595-4d85-b070-516222524d37 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.591498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/203] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/role_assignments?user.id=16ad9b32ecc740bcb9ef63fad18c96c2&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=798530c22da24d42b101d2f46d4939bb => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.638866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.639807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.644507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.644991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.645955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:30.646407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.648018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.657730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 99/204] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.664068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.673944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.815137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7554bcc-923a-409b-ac4a-ffcf1e9d26e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.867710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.868628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.868912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.869186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] PATH_INFO: `/v3/users/514acb994818468099c0b1935c905f80` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.876818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] RBAC: Authorizing `identity:get_user(user_id=514acb994818468099c0b1935c905f80)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.878321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.880387 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/205] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/users/514acb994818468099c0b1935c905f80 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.885937 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.020407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc577-f611-4ced-8ef9-c0b8ea67f9a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.044108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.072171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.072753 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.073036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.073313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.164930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.166377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.311015 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 100/206] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:31.316522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.398698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.425527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a038d9-5e7b-46e2-bf90-19d96f9df33e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.461120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.461996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.462282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.462562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.463787 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.465282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33aac6-d449-43e9-8bbd-6272f211c9dd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.467360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/207] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:31 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:31.473084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.500049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.631224 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ef03-723d-49e9-b925-65622a6b74ab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.677004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.677890 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.678340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.679394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.680455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, scope.project.id=37bb98420135461aaed594a67f3f07f5, user.id=514acb994818468099c0b1935c905f80) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:31.680905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.682499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136f1bb1-86d1-4eb4-b620-6eacc28c5191 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.692346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 101/208] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:31 2022] GET /identity/v3/role_assignments?user.id=514acb994818468099c0b1935c905f80&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:32.049167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867f9d-77d1-4854-9071-738c43081962 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.049640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867f9d-77d1-4854-9071-738c43081962 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.049640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47867f9d-77d1-4854-9071-738c43081962 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.051353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/209] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:32.059473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.059964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.060041 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.078141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:32.079323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:32.152287 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 102/210] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:32.176279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867f9d-77d1-4854-9071-738c43081962 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.291620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47867f9d-77d1-4854-9071-738c43081962 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.330476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.331194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.331479 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.331814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.342820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-39c4d5d7-a554-43bf-866d-b94c53119de1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:32.352097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.352957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:32.353726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:32.359310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/211] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.359447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.375465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.467090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a887afd-5e3f-45e9-929c-62e94e1645d0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.496784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebfcc-19ed-4514-a234-4d3c5559d078 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.497121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebfcc-19ed-4514-a234-4d3c5559d078 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.497353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebfcc-19ed-4514-a234-4d3c5559d078 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.499128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/212] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:32.504869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.505156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.505432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.511419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.513725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.514737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.515176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.515604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.516616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:32.517065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.518589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.525751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:32.527492 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:32.528664 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 103/213] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:32.541288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.611451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/214] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:32.637013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.640291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.672068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d6e6-7ba3-4c1a-991c-233ece1b8d35 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.710207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.711082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.711369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.711650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.712894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.714331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa2394-50df-44c5-810d-95bf549804c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.716141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 104/215] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:32.717380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585cde-0d7a-49b8-b85e-0cfe1b530c3f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.717669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585cde-0d7a-49b8-b85e-0cfe1b530c3f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.717950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5585cde-0d7a-49b8-b85e-0cfe1b530c3f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.719461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 105/216] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:32.725007 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5585cde-0d7a-49b8-b85e-0cfe1b530c3f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.748885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62928d9-08af-417b-9dd2-9ab1c6c1e9d5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.783247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.792324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.793151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.793415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.793691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.804999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-477cb670-4dcf-44fb-b56a-f81de04edfa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:32.812635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.813488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:32.814293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/217] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.815412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.815504 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.818447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.834593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:32.836613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:32.844406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5585cde-0d7a-49b8-b85e-0cfe1b530c3f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.879483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.880699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.880990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.881270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.883224 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.885062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.887102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 106/218] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:32.889131 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.904360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.912560 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/219] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:32.923490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.005395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c1f29-070c-46db-a46e-096a86769afb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.032755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e542a-bd9b-484c-95ab-98fdf92e5d87 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.041421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.042480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.042967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.043555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.044563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.045114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.046924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.055839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.058426 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 107/220] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.059554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.072927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.073822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.074111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.074386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/users/16ad9b32ecc740bcb9ef63fad18c96c2/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.076760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, user_id=16ad9b32ecc740bcb9ef63fad18c96c2, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.078487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.092400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/221] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:32 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/users/16ad9b32ecc740bcb9ef63fad18c96c2/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:33.093359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.156691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.192646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069abc4-770b-4574-8af6-8c13e2d61ba4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.231104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afaada-3d76-429e-80a9-33046b9b52a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.232164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06927280-7d89-401f-a689-095874f12b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.233101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927280-7d89-401f-a689-095874f12b6a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.233392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927280-7d89-401f-a689-095874f12b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.233788 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927280-7d89-401f-a689-095874f12b6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.235074 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06927280-7d89-401f-a689-095874f12b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.236665 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06927280-7d89-401f-a689-095874f12b6a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.238610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 108/222] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.240081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b9ee-378c-4824-982c-8913f83050a0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.240368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b9ee-378c-4824-982c-8913f83050a0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.240644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b9ee-378c-4824-982c-8913f83050a0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.246506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 109/223] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:33.246506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907b9ee-378c-4824-982c-8913f83050a0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.271768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.273265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.273554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.273829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] PATH_INFO: `/v3/users/deddc5d5f4874b99ac014584a89d5322` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.289323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] RBAC: Authorizing `identity:get_user(user_id=deddc5d5f4874b99ac014584a89d5322)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.290872 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2282-9bcd-4eaa-818d-a1f2d3a9ea4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.293029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/224] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/users/deddc5d5f4874b99ac014584a89d5322 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.294331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.294623 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.294901 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.314714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:33.315906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:33.361400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.377325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907b9ee-378c-4824-982c-8913f83050a0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.400877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/225] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:33.401738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.422154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.423310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.424111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.424611 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.425536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.427831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.438375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.453262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 110/226] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:33 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:38:33.454228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.528710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f32955-e714-4884-937d-faf7cccf9fdc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.567642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.568502 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.568805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.569614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.579887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00538874-6902-4a25-9d0d-1f4895c358f4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.614830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.621369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.622161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.622515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.622716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.636238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:33.645088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:33.645965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:33.646781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 111/227] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:33.651000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.652523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.653998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.730580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.779463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/228] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:33.784745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.785556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4461e-f746-4f66-bb04-5372d9141d9e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.831977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.832686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.832970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.833247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.834014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.834322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.837089 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.848346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 112/229] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:33.854836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.882000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.922123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7eb9a-91ad-4178-85b3-2a547eb21fba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.963897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.964809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.965087 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.965356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.966585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.968573 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.970742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/230] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:33.978004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.993009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8e385-f48d-4a06-8fe9-6f6d5eef2bbc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:34.022015 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:34.042440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:34.044554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.045048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.045524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.048525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:34.053294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:34.057492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 113/231] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:34.065433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:34.205249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a81c-98a4-413a-9e49-494e1f12fb4b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:34.206051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978ecd1-fd54-425d-8c74-d2849c0b8930 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:34.246275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:34.247479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.248021 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.248799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.250374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:34.252076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:34.254669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 114/232] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:34 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:34.261023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:34.277662 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:34.278806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.279299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.279815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.280936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=deddc5d5f4874b99ac014584a89d5322) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:34.281469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:34.283203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1ada3-b103-43ba-b311-db36af5152ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:34.299129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/233] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/role_assignments?user.id=deddc5d5f4874b99ac014584a89d5322&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:34.356333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:34.433010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f997f8a-8dd4-4761-837e-a6605fbd653c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:34.482715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:34.483615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.483897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.484209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/users/16ad9b32ecc740bcb9ef63fad18c96c2/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.490062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, user_id=16ad9b32ecc740bcb9ef63fad18c96c2, role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:34.493016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509ed4-daee-4e64-8184-f23652dffe01 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:34.520183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 115/234] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:34 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/users/16ad9b32ecc740bcb9ef63fad18c96c2/roles/798530c22da24d42b101d2f46d4939bb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:35.590019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea01bf-9ae7-4535-8838-de6945ce3151 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.590516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea01bf-9ae7-4535-8838-de6945ce3151 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.590589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea01bf-9ae7-4535-8838-de6945ce3151 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.592084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/235] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:35.597443 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.598224 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.598749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.616512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:35.617990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:35.710109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 116/236] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:35.733724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea01bf-9ae7-4535-8838-de6945ce3151 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.737004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:35.873571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea01bf-9ae7-4535-8838-de6945ce3151 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:35.909361 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:35.910578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.911068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.911539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.925162 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:35.934131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:35.935067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:35.935856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:35.938715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/237] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:35.945295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.001516 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.059780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb2daf-de54-41f6-8bf3-9d31ea8b8af7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.095371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.096078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.096367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.096649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.097408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:36.097721 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.099117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d001-a769-4d94-99e5-e4adf3816179 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.108633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 117/238] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:36.114262 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.181548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1e329-2d50-40db-9a6b-ddbf49e70ac5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.181838 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1e329-2d50-40db-9a6b-ddbf49e70ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.182030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1e329-2d50-40db-9a6b-ddbf49e70ac5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.183410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 118/239] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:36.203943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.203943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.203943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.224321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.236143 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:36.237360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:36.286825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e07f-0d9b-459b-834a-5a4caca189b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.370132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.371025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.371309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.371589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.375180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.376692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.384118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/240] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:36.385004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.387407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 119/241] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:36.410646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.500677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbad4c-e368-4861-9a36-e6bd70b5d6d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.530034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.548750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.562327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.562984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.563267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.563535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.621908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c0ad7-b535-49d4-9f09-3be436c83141 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.643519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.645286 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.668987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.669791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.670076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.670348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.684162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bc8473c9-2ced-49d1-8569-68afafb65ccc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:36.693369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:36.698259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 120/242] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:36.700432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dde0-09e2-4dd3-83d0-adb88a99b27f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.701116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dde0-09e2-4dd3-83d0-adb88a99b27f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.701116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dde0-09e2-4dd3-83d0-adb88a99b27f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.701116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 121/243] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:36.703307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.703596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.703875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.721776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:36.723823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:36.782679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/244] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:36.783459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.806183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 122/245] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:36.807942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.828265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.855811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.925064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687e830-ea35-4734-96a4-33e2bb51404b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.950396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc69e78-352c-4465-b339-e00c34ab8b37 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.970201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.970876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.971161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.971436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.972226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:36.972532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.974379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.984287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/246] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:36.985396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.003740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.005146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.005438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.005720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.008142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.010285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.012052 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 123/247] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.013717 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.090290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.099798 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235cd6-0149-4fcf-a802-009b36406c0e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.136796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.137624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.137902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.138187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.139387 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.140771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.140882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.142634 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/248] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.143540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.152687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd5a87-0898-45a6-b2cf-d2ef5c3e043c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.189773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.190644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.190929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.191203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] PATH_INFO: `/v3/users/a4f133b5f1964806a0e150967cffc069` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.205271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f133b5f1964806a0e150967cffc069)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.206745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6ef27-98ac-400e-b947-eeb6286ef0f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 124/249] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/users/a4f133b5f1964806a0e150967cffc069 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2025c9-c172-49b2-b602-a2684d4e4e07 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2025c9-c172-49b2-b602-a2684d4e4e07 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2025c9-c172-49b2-b602-a2684d4e4e07 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 125/250] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:37.213303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2025c9-c172-49b2-b602-a2684d4e4e07 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.266046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9406c-0519-47ef-93ad-fe7433614a23 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.304763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.305427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.305711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.305990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.306942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=16ad9b32ecc740bcb9ef63fad18c96c2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:37.307289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.308838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.323946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/251] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/role_assignments?user.id=16ad9b32ecc740bcb9ef63fad18c96c2&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.323946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.334345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2025c9-c172-49b2-b602-a2684d4e4e07 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.351785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.396857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.398378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.399044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.399427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.400951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.402907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b27e8-7288-4ef5-8890-c491daa9b4ff None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.432943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 126/252] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:37.434566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257ad8-4a3b-4a75-a1d3-b537a35d4430 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.434934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257ad8-4a3b-4a75-a1d3-b537a35d4430 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.435288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257ad8-4a3b-4a75-a1d3-b537a35d4430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.457326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ce257ad8-4a3b-4a75-a1d3-b537a35d4430 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:37.458715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce257ad8-4a3b-4a75-a1d3-b537a35d4430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:37.462648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.474225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e38-8817-49a4-8112-3ae737a04441 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.531600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.532912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.533208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.533493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.559536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 127/253] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:37.560824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d5d28-b313-447a-a540-751ad2ee4276 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.561156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d5d28-b313-447a-a540-751ad2ee4276 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.561388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d5d28-b313-447a-a540-751ad2ee4276 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.563250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 128/254] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:37.568899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.569284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.570264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.596275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:37.597475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:37.624876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.627248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.665150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.693056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 129/255] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:37.693985 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.760960 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/256] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:37.761838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.778573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.817515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9f9a2-8d51-4f84-af26-7db318c7e848 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.865432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.867510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.867810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.868121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.882878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:37.887251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf087e22-0ca8-44aa-81dd-eeb04019a472 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:37.892933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:37.893817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:37.894587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 130/257] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:37.895241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.925081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.940804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.941781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.942078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.942356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.956565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:37.966499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:37.967385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:37.968220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:37.969679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/258] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:37.970557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.039548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24b94-3fc1-48b0-b01b-69d9ad57eb61 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.069833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.080646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.081628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.081916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.082650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.083753 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.086317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.088844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 131/259] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.089706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.097958 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be8aa1-71d4-4603-a81c-d8e64faf6c9f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.145696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.146302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.146588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.146867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.148003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.148372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.149797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.159128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/260] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.159993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.215153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.218262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a712a4-85ac-43a0-8c78-77e5c14e3195 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.253936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.254941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.255421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.255943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.257502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.258011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.259584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.268090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 132/261] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.269190 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.272886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985add-acd9-40ee-8efe-8de341f7e2a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.309739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.310403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.310688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.310965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.311860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=a4f133b5f1964806a0e150967cffc069) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.312215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.313833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.327300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/262] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/role_assignments?user.id=a4f133b5f1964806a0e150967cffc069&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.328394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.333917 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.385246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f47e6-c72b-4e6c-9f48-7a18d21f7103 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.424202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.425531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.426016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.426502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] PATH_INFO: `/v3/users/deddc5d5f4874b99ac014584a89d5322` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.428269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=deddc5d5f4874b99ac014584a89d5322)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.430815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.433152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 133/263] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/users/deddc5d5f4874b99ac014584a89d5322 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.438797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.512927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf097-e548-4384-b153-590969c36300 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.524796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.549946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.550940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.551184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.551469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] PATH_INFO: `/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.552835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] RBAC: Authorizing `identity:get_user(user_id=16ad9b32ecc740bcb9ef63fad18c96c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.554982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.556935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/264] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/users/16ad9b32ecc740bcb9ef63fad18c96c2 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.565014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.567654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e6d08-4e08-440e-b995-694484c10fb7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.598280 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.609911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.611350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.611862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.612383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.613878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.615722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.619117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 134/265] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.629034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.711200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45165f-473e-4de9-8588-902e50a1065a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.757691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.758396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.758689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.758969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.762279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07895f-ae26-4a19-8953-71fe2310e7ae None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.842180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.843089 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.848540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.848851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/users/deddc5d5f4874b99ac014584a89d5322/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.851958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, user_id=deddc5d5f4874b99ac014584a89d5322, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.855620 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.865329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.885762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 135/266] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:38 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/users/deddc5d5f4874b99ac014584a89d5322/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:38.896878 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.898663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.043491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/267] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:39.049144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.134910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.186534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9c3e-cfe4-4fd2-ac57-c7ef1f075b15 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.222319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.223185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.223484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.223744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.224970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.226930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6bd46-3260-4588-9e71-a002c1e3b95d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.232616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 136/268] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:39.234892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.333550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.368409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5bb6d-b3ba-408d-a425-671d6161afd1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.409564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.410228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.410510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.410784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.411616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=16ad9b32ecc740bcb9ef63fad18c96c2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:39.411945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.413468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d31e-1888-4962-8373-56d5d7b541b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.424046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/269] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/role_assignments?user.id=16ad9b32ecc740bcb9ef63fad18c96c2&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=798530c22da24d42b101d2f46d4939bb => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.056944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2196-4389-455c-b72a-bd8011bc471b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.057445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2196-4389-455c-b72a-bd8011bc471b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.057522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a2196-4389-455c-b72a-bd8011bc471b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.058880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 137/270] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:40.064218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.064607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.064799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.085011 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:40.086379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:40.136341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ae2c-7773-4ca3-ba84-c31b11287ec0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.136634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ae2c-7773-4ca3-ba84-c31b11287ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.136911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ae2c-7773-4ca3-ba84-c31b11287ec0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.138415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 138/271] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:40.143218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.143507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.143790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.162431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:40.163977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:40.165879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/272] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:40.189326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.237122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 139/273] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:40.260970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.265516 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.292118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb115b-f9c0-425f-a780-04a4d2e8f2c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.325331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.327669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.328502 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.328786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.329142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.340122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:40.348010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:40.348906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:40.349678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/274] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:40.357737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.389981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c853dc4-17de-4f7c-bf57-3e68ff7b8e76 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.429218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.430440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.430905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.431359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.432746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.434381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.440986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 140/275] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.442153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.463201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11120c8a-19db-4cc2-ab0c-46fbb23ca554 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.493740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.498902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.500072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.500677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.501287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.502131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.502509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.504303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.518948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/276] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:40.520313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.562473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a43d3-2cc7-40a2-bf25-909016bf1889 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.585910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.601407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.602084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.602370 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.602646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.603363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.604834 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.628443 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 141/277] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:38:40 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:40.634417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c35a0-ccb5-4190-a37e-bbdb3abb23f5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.674728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.675945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.676408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.676886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] PATH_INFO: `/v3/users/1f18a1044b9a47df9397e8472692e72f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.693393 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=1f18a1044b9a47df9397e8472692e72f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.695347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4a8c-49cc-4a5c-b58e-92b023bbe3d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.702349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/278] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/users/1f18a1044b9a47df9397e8472692e72f => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:40.704142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.826954 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daea6a-ea47-472f-984e-846fc274d773 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.866711 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.867676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.868163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.868641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.869686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, user.id=1f18a1044b9a47df9397e8472692e72f, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.870190 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.871795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b22da-dd4a-4ddf-8517-7baf729370dc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.880073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 142/279] 10.222.0.45 () {56 vars in 1308 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/role_assignments?user.id=1f18a1044b9a47df9397e8472692e72f&scope.system=all&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:41.006007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3ccc-f5f6-4840-b864-15fedcd85afd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.006861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3ccc-f5f6-4840-b864-15fedcd85afd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.007265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c3ccc-f5f6-4840-b864-15fedcd85afd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.008786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/280] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:41.014528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.015115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.015615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.037091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:41.039420 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:41.083053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.128250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 143/281] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:41.150995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c3ccc-f5f6-4840-b864-15fedcd85afd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.275345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.356573 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c3ccc-f5f6-4840-b864-15fedcd85afd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.409699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.410872 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.411368 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.411840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.423199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:41.431911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:41.432839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:41.433607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:41.436289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/282] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:41.444031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.591948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b896db-15c7-4644-b3ec-b3d8776d2e52 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.627886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.628945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.629453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.629927 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.630877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:41.631502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.633196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e116a0-97b6-4ef9-b6e8-1ed5e85e8861 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.642045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 144/283] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:41.648817 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.662920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfa00-62eb-4ba5-abbd-9966d784773f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.663213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfa00-62eb-4ba5-abbd-9966d784773f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.663497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfa00-62eb-4ba5-abbd-9966d784773f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.666249 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 145/284] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:41.670326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.670615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.670892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.700919 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:41.702109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:41.774758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.785388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43c3e-f97f-4b57-aa76-883635971090 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.808282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.847519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 146/285] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:41.865288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.866438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.866727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.867005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] PATH_INFO: `/v3/users/a4f133b5f1964806a0e150967cffc069` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.868274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f133b5f1964806a0e150967cffc069)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.869735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.871660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/286] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/users/a4f133b5f1964806a0e150967cffc069 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:41.876829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.983731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd217-7c5e-4af5-afee-b2b50e4dc3fc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.021150 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.023252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.024847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.025993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.028195 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.030852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.034438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 147/287] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:42.043956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.096403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207afbf-4a0f-44b6-8f3e-a96f0828d0f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.101630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.164755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.165571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.165850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.166120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.187490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:42.198469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:42.199414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:42.200223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:42.200828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/288] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:42.206005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.239577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f161d-3211-4b93-a147-837604f0f1be None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.290819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.310268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.311170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.311454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.311730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/users/a4f133b5f1964806a0e150967cffc069/roles/efeb4abc22fe4ff19bcabe767842fc2d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.314850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, user_id=a4f133b5f1964806a0e150967cffc069, role_id=efeb4abc22fe4ff19bcabe767842fc2d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.316318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.342956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 148/289] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:38:42 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/users/a4f133b5f1964806a0e150967cffc069/roles/efeb4abc22fe4ff19bcabe767842fc2d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:42.379264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2c683-3e08-4e5d-be9b-fcfa438b8303 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.416506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.417374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.417659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.417933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.418767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:42.419080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.420567 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.430098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/290] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:42.437187 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.552811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.566059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859049f-61b5-464b-841e-9d2e6c83538c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.609158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.610040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.610327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.610606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] PATH_INFO: `/v3/users/deddc5d5f4874b99ac014584a89d5322` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.612870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] RBAC: Authorizing `identity:get_user(user_id=deddc5d5f4874b99ac014584a89d5322)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.614367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.616279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 149/291] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/users/deddc5d5f4874b99ac014584a89d5322 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:42.621884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.743112 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a077525-d99a-4d19-8f4a-b285ed950099 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.744803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.781300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.781956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.782239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.782519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.854622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.856133 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.960185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/292] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:42.965332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.096246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbf41c-a3d8-4528-999e-31fcde7079c4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.134807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.135592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.135878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.136183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.137446 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.138879 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5278bdc-ed22-4a96-9b39-6bd2d458bf8d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.141091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 150/293] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:43.149126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.151469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.300547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8eaea-86c7-4b8c-8c70-e3972429eb86 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.359959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.359959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.359959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.359959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.360369 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=deddc5d5f4874b99ac014584a89d5322) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:43.360369 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.361861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e1568-e286-4d5f-8192-a6f93d212afa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.373267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/294] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/role_assignments?user.id=deddc5d5f4874b99ac014584a89d5322&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:43.481948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13712f9-be69-4e6f-961b-9bbf5d255f44 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.481948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13712f9-be69-4e6f-961b-9bbf5d255f44 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.481948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13712f9-be69-4e6f-961b-9bbf5d255f44 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.482648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 151/295] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:43.487316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.487601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.487882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.512341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:43.513613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:43.566513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e60ff-ad5b-4a2c-b9e2-cb8faf5069f5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.567145 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e60ff-ad5b-4a2c-b9e2-cb8faf5069f5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.567630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e60ff-ad5b-4a2c-b9e2-cb8faf5069f5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.569958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 152/296] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:43.574985 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebdae8-01e1-4e6c-98a0-3bec55b8de16 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.575586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebdae8-01e1-4e6c-98a0-3bec55b8de16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.576117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebdae8-01e1-4e6c-98a0-3bec55b8de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.603251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/297] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:43.604329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.606370 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0cebdae8-01e1-4e6c-98a0-3bec55b8de16 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:43.607737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0cebdae8-01e1-4e6c-98a0-3bec55b8de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:43.625709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.652410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.722683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 153/298] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:43.724440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b332-4c5b-43f2-b4ae-657b2154db16 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.725081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b332-4c5b-43f2-b4ae-657b2154db16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.726136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34b332-4c5b-43f2-b4ae-657b2154db16 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.728236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 154/299] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:43.734951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.735456 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.736223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.755244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:43.756798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:43.759332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81367d3-426f-4a81-b5a2-ca7039875051 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.796780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.797578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.797862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.798135 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.799644 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.800756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.802526 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/300] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:43.805059 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.827689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 155/301] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:43.828551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.880893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.925038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2bc67-8c66-46c8-92c7-1e9aa3b6583d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.925223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.941840 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaef85-903a-4436-88db-962b9ee5ec1f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.961434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.962254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.962539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.962810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.973435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:43.978677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.979345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.979633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.979911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] PATH_INFO: `/v3/groups` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.980637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:43.981733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:43.982556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:43.983336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:43.983951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.984024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/302] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:43.984024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.001470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 156/303] 10.222.0.45 () {60 vars in 1134 bytes} [Wed May 25 13:38:43 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:44.002226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.122464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac5481-436b-4661-b47c-27ce99b9371b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.135192 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330864-202a-41f7-880c-e98737dce8da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.156685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.174134 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.175819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.176345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.176932 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.180580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.181258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.181544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.181816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.182578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.182882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.184299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.194370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:44.195974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 157/304] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.201902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:44.204035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.204905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:44.205782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:44.206373 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/305] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.211860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.218029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.329094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32caa-6682-45e8-8e62-c9807008ecfc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.342167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f716031-4073-456d-be0b-7c511ee70443 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.375608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.376800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.377093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.377372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] PATH_INFO: `/v3/users/1f18a1044b9a47df9397e8472692e72f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.379151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] RBAC: Authorizing `identity:get_user(user_id=1f18a1044b9a47df9397e8472692e72f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.381044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.386953 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.387879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.388257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 158/306] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/users/1f18a1044b9a47df9397e8472692e72f => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.388632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.389110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.389742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.390449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.391004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.392645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.402291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/307] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:44.407258 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.457585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.481637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.515940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efe6a6-4c14-40d6-9855-4bca4a3b4cf3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.517036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f53f-4587-47db-9bdf-edf3e2d24e1d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.553823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.555287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.555572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.555845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] PATH_INFO: `/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.557160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.558465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.558999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.559494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] PATH_INFO: `/v3/system/users/1f18a1044b9a47df9397e8472692e72f/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.561235 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f18a1044b9a47df9397e8472692e72f, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.562836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.571845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] RBAC: Authorizing `identity:get_user(user_id=36c6fdcd9a76432bb27526fbb9fc42ee)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.573660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f3b9b-6d0d-41f4-8950-cc5cc63eb707 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.576099 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 159/308] 10.222.0.45 () {58 vars in 1334 bytes} [Wed May 25 13:38:44 2022] PUT /identity/v3/system/users/1f18a1044b9a47df9397e8472692e72f/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:44.580173 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/309] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:44.581537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.706104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c417d5-12b7-457e-a4ec-dc21f860a68c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.746405 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.747555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.748090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.748571 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.749629 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, user.id=36c6fdcd9a76432bb27526fbb9fc42ee, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.750154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.751868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978aedd-e38b-4412-9e3c-9604395f4463 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.759255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 160/310] 10.222.0.45 () {56 vars in 1308 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/role_assignments?user.id=36c6fdcd9a76432bb27526fbb9fc42ee&scope.system=all&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.191184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0a6f9-d938-4315-997f-4fabd8a7218b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.191719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0a6f9-d938-4315-997f-4fabd8a7218b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.192245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0a6f9-d938-4315-997f-4fabd8a7218b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.194692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/311] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:45.200385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.200824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.200996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.208062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.221417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:45.222654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:45.297915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 161/312] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:45.322089 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0a6f9-d938-4315-997f-4fabd8a7218b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.431869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0a6f9-d938-4315-997f-4fabd8a7218b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.434343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.468922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.469635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.469921 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.470197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.482191 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:45.490214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:45.491132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:45.491945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/313] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:45.497611 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.609921 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c052bc7-f0e0-49c0-bd72-824e7842efb0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.645668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.646397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.646683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.647576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.648414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:45.648724 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.650101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.651481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.659308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 162/314] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.665136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.771226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb37599-e2a3-4d4c-819a-16033f1ed76e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.809380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.810261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.810545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.810825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] PATH_INFO: `/v3/users/a4f133b5f1964806a0e150967cffc069` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.812071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] RBAC: Authorizing `identity:get_user(user_id=a4f133b5f1964806a0e150967cffc069)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.813543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.815464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/315] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/users/a4f133b5f1964806a0e150967cffc069 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.821002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.933216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0b10e-96cc-42e6-a834-c399235cf9d3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.969566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.970283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.970569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.970847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.036760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.043798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.045287 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.148243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 163/316] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:46.153693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.189551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.264173 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeec71d-b7c2-4fd4-9f52-aded0117535c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.299557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.300386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.300815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.301095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.302310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.303867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3cf50-8d46-4da0-af1f-140039d3628a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.305887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/317] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:46.311743 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.383568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b92d-94f4-4c1e-9c7f-e4d9f31ae130 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.383901 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b92d-94f4-4c1e-9c7f-e4d9f31ae130 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.384160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14b92d-94f4-4c1e-9c7f-e4d9f31ae130 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.386251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/318] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:46.392787 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.393338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.393906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.417622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.418054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:46.420570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:46.424111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83c230-1bfb-4110-a167-9968531f9b2c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.462938 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a281-0618-4190-824d-795769be508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.464002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a281-0618-4190-824d-795769be508c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.464532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a281-0618-4190-824d-795769be508c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.465044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a281-0618-4190-824d-795769be508c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.466027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8a281-0618-4190-824d-795769be508c None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, scope.project.id=f5f8d2d683da441baa1480f203594842, user.id=a4f133b5f1964806a0e150967cffc069) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:46.466633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8a281-0618-4190-824d-795769be508c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.468261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8a281-0618-4190-824d-795769be508c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.476042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 164/319] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/role_assignments?user.id=a4f133b5f1964806a0e150967cffc069&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:46.510939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/320] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:46.535097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a281-0618-4190-824d-795769be508c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.655281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a281-0618-4190-824d-795769be508c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.696059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.696868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.697153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.697429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.698634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.700124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca00c4-561f-4abe-a0e5-d20d19a767be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.701914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 165/321] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:46.707978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.747464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.829541 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbf6a-3dc3-4fdc-a867-2250b90c1254 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.864347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.865013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.865299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.866103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.866796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.868266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa0c38-afd9-4126-8862-2b0d52e69dc3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.891963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/322] 10.222.0.45 () {60 vars in 1134 bytes} [Wed May 25 13:38:46 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:46.939305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ea77-db00-44c6-b509-bd42e110d3b1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.939494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ea77-db00-44c6-b509-bd42e110d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.939779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ea77-db00-44c6-b509-bd42e110d3b1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.941414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 166/323] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:46.948205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.948513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.948774 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.967216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:46.968558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:46.991135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42752e-ec16-439f-8aa2-7063d811a06e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.991447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42752e-ec16-439f-8aa2-7063d811a06e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.991696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42752e-ec16-439f-8aa2-7063d811a06e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.993303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 167/324] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:46.999033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.999621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.000130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.024742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:47.025972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:47.049327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/325] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:47.052978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.073835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.098752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.109205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 168/326] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:47.132494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.189138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9eef0-b513-4ae7-a4e8-69cea064929b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.235606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.236358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.236797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.237211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.251781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:47.259582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ac7e8-e847-4134-8248-8409978aa122 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:47.261571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:47.262637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:47.263600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:47.265699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/327] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:47.272512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.309118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.309987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.310440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.310932 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.324136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:47.333473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:47.335115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:47.336027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 169/328] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:47.342290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.367587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.394648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.432401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b77c-f134-4b44-8047-89b058618076 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.469510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee526-adcd-43ad-a1dd-82b06fc3858c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.477278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.478474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.478834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.479185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.480078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.480824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.482667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.495156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/329] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:47.500866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.514454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.515475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.516019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.516496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.517535 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.518039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.519614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.529916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 170/330] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:47.535701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.635362 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352ecf6-0977-412d-9360-0343c9c6c24d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.641775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.667676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9b1d-78fe-4e4a-80a7-1f8d8c0134ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.683408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.687519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.688146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.688624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] PATH_INFO: `/v3/users/1f18a1044b9a47df9397e8472692e72f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.690197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] RBAC: Authorizing `identity:get_user(user_id=1f18a1044b9a47df9397e8472692e72f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.691875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.694742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/331] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/users/1f18a1044b9a47df9397e8472692e72f => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:47.703671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.711299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.712218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.712504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.712782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] PATH_INFO: `/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.714803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] RBAC: Authorizing `identity:get_user(user_id=36c6fdcd9a76432bb27526fbb9fc42ee)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.716841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.719269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 171/332] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:47.720923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.726588 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.842714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c807a5-6037-480a-9c67-5b1a92665225 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.869239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da61b9-11ca-4c36-89de-de5ca197459b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.878604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.879267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.879555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.879835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.880710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] RBAC: Adding query filter params (role.id=eda7e112a67a4f12993c9af8f1f3b587, user.id=1f18a1044b9a47df9397e8472692e72f, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.881028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.883156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9f2ef-2326-4319-a481-4f82fc7f030e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.900312 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/333] 10.222.0.45 () {56 vars in 1308 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/role_assignments?user.id=1f18a1044b9a47df9397e8472692e72f&scope.system=all&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:47.922904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.924245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.924737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.925314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] PATH_INFO: `/v3/system/users/36c6fdcd9a76432bb27526fbb9fc42ee/roles/efeb4abc22fe4ff19bcabe767842fc2d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.932988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=36c6fdcd9a76432bb27526fbb9fc42ee, role_id=efeb4abc22fe4ff19bcabe767842fc2d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.934771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f18cb-48cc-4fc9-bff0-d1f7342cde85 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.955484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 172/334] 10.222.0.45 () {58 vars in 1334 bytes} [Wed May 25 13:38:47 2022] PUT /identity/v3/system/users/36c6fdcd9a76432bb27526fbb9fc42ee/roles/efeb4abc22fe4ff19bcabe767842fc2d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:38:49.940052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeba7-d2d2-471f-9b3d-02495289e2cb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.940895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeba7-d2d2-471f-9b3d-02495289e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.941322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eeba7-d2d2-471f-9b3d-02495289e2cb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.943105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/335] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:49.949248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.949741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.949981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.972995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:49.974218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:49.983708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.041860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dff24-f3db-435b-bb41-37689ec1af75 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.042634 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dff24-f3db-435b-bb41-37689ec1af75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.043118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dff24-f3db-435b-bb41-37689ec1af75 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.045137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/336] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:50.053700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.054290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.054791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.069408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 173/337] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:50.085689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:50.086990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:50.134831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.191308 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.209305 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/338] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:50.245711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.313985 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1ad8-6eca-4ea4-b8d9-9f7b343ae2fa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.389636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.390467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.390750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.391045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.404370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21b636-38cb-4b45-a8f1-02e7fc6aaf7b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.419147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:50.436189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:50.437170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:50.437991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 174/339] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:50.450135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.456095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.457845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.458859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.459965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.460574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.485036 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:50.494979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:50.495948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:50.496752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:50.503965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/340] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:50.525475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.613360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb8680-2321-4e44-aa1c-73850b0f17b1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.644513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.653031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.653710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.653997 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.654276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.655039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.655348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.660540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.667359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 175/341] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.681031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.706276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f58de-ec76-4483-b781-d79477ea7d18 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.743890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.744848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.745270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.745679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.746585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.747030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.748606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.759966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/342] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.768028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.811156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.881022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20052093-99bb-4c2c-9b5d-6e58099857b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.935231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.936506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.937689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.938347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.980724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7d64a-911f-44d4-b875-bdf07d72b409 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.024838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.048714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.050759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8009b7e-6976-43be-9322-4843172727e2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.051942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.053571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.054110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.054635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.118518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.194320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.196058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c48d4-d49f-4772-8765-031b16193247 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.221549 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 176/343] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:51.223248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01342-07a5-4e14-9c8e-b9bc1a37f883 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.223740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01342-07a5-4e14-9c8e-b9bc1a37f883 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.224288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01342-07a5-4e14-9c8e-b9bc1a37f883 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.226527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 177/344] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:51.232020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2f82-4bc7-4e2a-83a7-6a39a360c523 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.232604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2f82-4bc7-4e2a-83a7-6a39a360c523 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.233085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee2f82-4bc7-4e2a-83a7-6a39a360c523 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.234648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 178/345] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:51.240191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.241330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.241811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.264083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:51.265292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:51.365199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/346] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:38:51.367456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.367988 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.368515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.391894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:51.393607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:51.399083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 179/347] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:51.399083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.422724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.502751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/348] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:51.504334 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.519346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.529327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15cbba-47a8-4d41-8c9d-51d07638e2a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.571410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.573075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.573372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.573710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.575251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.576916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.578605 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 180/349] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.579587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.648268 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08e85b-be6e-43b2-8135-8cacac12c96d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.696067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.697190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.697481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.697761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.699663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.701904 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05367ee8-f587-4da7-b39a-6798bbf952c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.704393 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/350] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.705902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be14e-8f82-4989-9ad9-920fa79f7539 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.706197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be14e-8f82-4989-9ad9-920fa79f7539 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.706478 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be14e-8f82-4989-9ad9-920fa79f7539 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.707864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/351] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:51.708546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42be14e-8f82-4989-9ad9-920fa79f7539 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.710542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755c320-a9ea-4a81-92ff-166949392845 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.732989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.757800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.758524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.758805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.760540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.778225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:51.786105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:51.787486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:51.788532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 181/352] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:51.794763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.831547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.856963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42be14e-8f82-4989-9ad9-920fa79f7539 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.899158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.899994 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.900290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.900564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.912556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bca179e0-0d72-48c6-959c-b6ac7b566af1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:51.922860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:51.923794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:51.924617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/353] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:51.925849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.925849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.925849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.939579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5a03f-5957-47a3-917a-8b07fd13b890 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.945990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:51.947175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:51.983253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.984199 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.984480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.984918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.992411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.994066 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.995906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 182/354] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.996877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.026587 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/355] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:52.028072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.046384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.111087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.134302 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70088a43-b7a4-421b-bcc7-a5db4313d9e4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.169770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848b55c-a1c3-44e4-bf69-e811fd40de2a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.182893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.183854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.184519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.184927 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.186741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.187518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.189640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 183/356] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.190693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.221287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.221984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.222273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.222551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.223327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:52.223643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.226118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.236430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/357] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.237303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.325527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.332527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1b346-d66a-446d-a1a3-fe366a391626 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.377926 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a4dee-cc95-4f5b-85ea-997c163fa7d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.384216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.384899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.385185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.385952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.386719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:52.387033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.388858 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.400643 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 184/358] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.401823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.419336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.420475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.420475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.420685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.421398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=37bb98420135461aaed594a67f3f07f5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:52.421709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.423175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.425484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.431026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/359] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=798530c22da24d42b101d2f46d4939bb => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.432023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.547742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153ebf5-f7a1-49f4-b10e-8263a374e830 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.565979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885567d-c511-4160-9dc0-cbef6f382a61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.603832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.604714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.605001 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.605475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.613328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.614010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.614302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.614583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.615599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=37bb98420135461aaed594a67f3f07f5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:52.618165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.618165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.620159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.622186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:52.625694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/360] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.626597 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:52.638792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:52.639802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:52.640639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:52.642273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 185/361] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:52.643589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.760347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.793867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723dd4d-8de0-4548-9b8c-05f157313caa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.808130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54448ee-cd44-4c9e-b143-7384808e4807 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:52.855601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.856210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.856491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.856766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.864897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:52.866672 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.867179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.867660 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] PATH_INFO: `/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.869142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] RBAC: Authorizing `identity:get_user(user_id=36c6fdcd9a76432bb27526fbb9fc42ee)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.871472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c501f-cbfd-4aaf-b708-9706a8f43207 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:52.873408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 186/362] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/users/36c6fdcd9a76432bb27526fbb9fc42ee => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.874859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9577bb4-1361-4d63-810c-6f8cef239ff3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.875371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9577bb4-1361-4d63-810c-6f8cef239ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.880590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9577bb4-1361-4d63-810c-6f8cef239ff3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.884367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 187/363] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:52.885180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9577bb4-1361-4d63-810c-6f8cef239ff3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:52.949115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:52.955267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:52.957078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c8f06-f508-4213-8474-19763fb1ed7f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.006727 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9577bb4-1361-4d63-810c-6f8cef239ff3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.046093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.046867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.047152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.047431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.048217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:53.048660 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.050040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.060164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 188/364] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:53.060968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.066906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/365] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:53.069555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.070072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.070542 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.077767 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.089650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:53.091170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:53.161091 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/366] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:53.162768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.178394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f7da3-99c7-41f9-aa1f-d563be1e62ed None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.208314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.219588 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.220286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.220574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.220855 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.221792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] RBAC: Adding query filter params (role.id=efeb4abc22fe4ff19bcabe767842fc2d, user.id=36c6fdcd9a76432bb27526fbb9fc42ee, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:53.222327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.223555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.240286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 189/367] 10.222.0.45 () {56 vars in 1308 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/role_assignments?user.id=36c6fdcd9a76432bb27526fbb9fc42ee&scope.system=all&role.id=efeb4abc22fe4ff19bcabe767842fc2d => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:53.241319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.288604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d1f9f-bd7e-4917-8a99-55193ccb0d9e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.326754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.327753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.328390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.328889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.370915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.406154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.407814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.438005 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363ad5-9c99-438f-8f2c-f50812d638b1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.460491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.510421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.511318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.511607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.511887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.516727 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.518185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.520322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 190/368] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:53.521179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.524926 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/369] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:53.526652 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.648375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc5e4-8901-4f7e-9273-efa45162e27e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.680772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.686444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.687246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.687528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.687803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.701364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:53.710070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:53.711029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:53.711795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 191/370] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:53.712484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.732246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df251-006a-41dd-bae5-8d7457f7ba5b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.767038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.778505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.779431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.779716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.780037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.781248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.782716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.784689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/371] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:53.785471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.825978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7410-d224-489e-ac52-abe42a9a2e46 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.863642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.864794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.865226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.865923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.868833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.870445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.873366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 192/372] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:53.881851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.905692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b94a4b-ffe2-48cc-b36e-ddb240761dbf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.940971 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.941884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.942009 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.942311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.942956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:53.943266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.945366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.954531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/373] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:53.955306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.980345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.030720 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.036857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34c005-e77b-4a84-821e-91c220e1add8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.082455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7e6ab-03f6-48e1-8706-2f2fe9f1cd71 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.087220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.088119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.088550 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.088989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.089986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=f5f8d2d683da441baa1480f203594842) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:54.090448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.092099 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.101441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 193/374] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.102652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.144169 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.145055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.145339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.145615 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.148815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.150329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.152103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/375] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.161119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.323119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011872f-7938-4671-ab57-589e6f464958 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.358600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8824040-465b-4fec-84c5-f4a9228862b9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.364685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.380634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.381402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.381689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.382110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.390664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.400934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.401789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.402190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.402466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.403356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=f5f8d2d683da441baa1480f203594842) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:54.403675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.405483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.411653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/376] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=798530c22da24d42b101d2f46d4939bb => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.496036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.497493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.630290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 194/377] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:54.636758 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.784410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe23781-bbf8-46f9-8cca-e2669b3d4d83 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.827445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.828588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.829016 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.829456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] PATH_INFO: `/v3/projects/c319a094752b41428929e5f2edeaa06e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.830846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] RBAC: Authorizing `identity:get_project(project_id=c319a094752b41428929e5f2edeaa06e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.832554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ed75-aae7-4314-847f-ff513a5b7908 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.834986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/378] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/projects/c319a094752b41428929e5f2edeaa06e => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.840730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.890542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.989936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d28f69-a6c3-44b0-8137-91e4f0bef78a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.991108 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d28f69-a6c3-44b0-8137-91e4f0bef78a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.991284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b431c-bf32-477d-bbff-8b7de1d710d9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.991775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d28f69-a6c3-44b0-8137-91e4f0bef78a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.993357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/379] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:54.998625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.999153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.999627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.020161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:55.022521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:55.037945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.038826 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.039109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.039385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] PATH_INFO: `/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.045041 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] RBAC: Authorizing `identity:get_group(group_id=ed8badf04c9c4a6192ca00a2ca41c1a8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.049720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.051860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 195/380] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.058062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.108410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.134446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/381] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:55.159247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.164448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.178970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567dc71-19fc-40c2-ade7-68e04c844db1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.216573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.217699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.217989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.218267 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.219219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] RBAC: Adding query filter params (group.id=ed8badf04c9c4a6192ca00a2ca41c1a8, role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=c319a094752b41428929e5f2edeaa06e) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:55.219542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.221020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.227949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 196/382] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/role_assignments?group.id=ed8badf04c9c4a6192ca00a2ca41c1a8&scope.project.id=c319a094752b41428929e5f2edeaa06e&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:55.330884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69adfff-4bb1-482c-a87f-2cc1281e0e2c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.374279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.375100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.375389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.375667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.387364 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4d960842-9749-422e-862e-f889a59628b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:55.395938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:55.396939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:55.397714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/383] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:55.403543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.527996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807933d-7bfd-476a-adac-08b5c0603989 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.528628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807933d-7bfd-476a-adac-08b5c0603989 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.529110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807933d-7bfd-476a-adac-08b5c0603989 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.531163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/384] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:55.536333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.536859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.537352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.551621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e06c0-d65c-408e-8e78-d793524507ca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.560751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:55.562331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:55.593150 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.594068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.594619 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.595057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.596085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:55.596531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.598186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 197/385] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.626993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.636279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.656687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.680033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/386] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:55.735531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.792128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1991d-a688-4a12-8f8d-81bc38388f69 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.834334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.835588 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.836167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.836593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.837940 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.839690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.841837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 198/387] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.846819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.939898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d75a95-60d6-4656-8b4d-e39f85e5502c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.944352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.980035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad759f6-72cc-4cd5-bfa3-e1b21dbced25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.017646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.019520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.020438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.021341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.023735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.026179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.026942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.027946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.030168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.030621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.031670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 199/388] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:56.038771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.062987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:56.071775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:56.072919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:56.073805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:56.076857 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/389] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:56.082103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.129192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.197200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa08e67-84fe-4c7a-b6c2-9ee6cc0e3879 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.231899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fe55-efc8-4242-876c-0c6ff6cb5583 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.257175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.257866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.258152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.258430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.259192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:56.259506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.260936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.268779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.269928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.270430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.270905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.272982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/390] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:56.273857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, group_id=ec5c9745c73a4ec787a7a9f03fdcf333, role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.275487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.278049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.297817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 200/391] 10.222.0.45 () {58 vars in 1442 bytes} [Wed May 25 13:38:56 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/798530c22da24d42b101d2f46d4939bb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:38:56.410509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.414438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6085ce6-4b8f-47be-96b4-97ad2680f036 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.463270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.464199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.464487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.464767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.468715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.470211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.472145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/392] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:56.478172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.508303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.609827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb029f38-4d80-49e9-abfc-b8cc23550313 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.661548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.663062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.663349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.663622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.668644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.670090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ddf23-df9e-4d0e-b582-53ae06d77f12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.672111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 201/393] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:56.677700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.843667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5871a4-1ead-4268-ad94-218860c40a15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.897346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.898156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.898444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.898723 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.900739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] RBAC: Authorizing `identity:create_grant(project_id=37bb98420135461aaed594a67f3f07f5, group_id=ec5c9745c73a4ec787a7a9f03fdcf333, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.902187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca9edc-2da2-4053-8566-81a69e12c056 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.929013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/394] 10.222.0.45 () {58 vars in 1442 bytes} [Wed May 25 13:38:56 2022] PUT /identity/v3/projects/37bb98420135461aaed594a67f3f07f5/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:57.203907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4442c-3b92-4b3f-9fc7-7eb0dc4e9578 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.203907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4442c-3b92-4b3f-9fc7-7eb0dc4e9578 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.203907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4442c-3b92-4b3f-9fc7-7eb0dc4e9578 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.203907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 202/395] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:57.210618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce027e6-bcb7-4836-b38c-d43abdc18e49 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.211417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce027e6-bcb7-4836-b38c-d43abdc18e49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.211859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce027e6-bcb7-4836-b38c-d43abdc18e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.235895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ce027e6-bcb7-4836-b38c-d43abdc18e49 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:57.237449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ce027e6-bcb7-4836-b38c-d43abdc18e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:57.272332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.333440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/396] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:57.368594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4442c-3b92-4b3f-9fc7-7eb0dc4e9578 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.458254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99eba0-8d56-4622-a09c-3d4895c6a30c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.458870 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99eba0-8d56-4622-a09c-3d4895c6a30c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.459511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99eba0-8d56-4622-a09c-3d4895c6a30c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.461200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.462780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/397] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:57.468236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.468763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.469297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.486316 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4442c-3b92-4b3f-9fc7-7eb0dc4e9578 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.490251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:57.491834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:57.522056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.523162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.523610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.524067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.535604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:57.544472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:57.545572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:38:57.546569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:57.548982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 203/398] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:57.561152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.571710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/399] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:57.595108 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.691818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.707268 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d124d-7ce2-44d7-ac02-275f6c92c00e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.723418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee99fb-7247-4b2d-9a94-40430fabfa25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.747130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.756501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.757169 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.757682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.769091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.770083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.770570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.771058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.773094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.773493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:57.773869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.775441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.784589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 204/400] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.791307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:57.791683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:57.796270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:38:57.797052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/401] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:57.797707 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.803949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.938549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856944f-eace-447e-953f-3d5f3776fa7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.979141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.979739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.980065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.980345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.981106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.981420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.982820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.996138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/402] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.997562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.010683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.045218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148051-c3e5-48b6-9332-4f2b92217c15 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.094916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.096166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.096596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.097016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.098909 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.101635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff67fa-2e4d-4b17-b8ae-cc72b239060e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.108496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 205/403] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:58.109415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fa109-8995-4fab-b191-2a36c080f396 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.109706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fa109-8995-4fab-b191-2a36c080f396 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.109987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fa109-8995-4fab-b191-2a36c080f396 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.111372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 206/404] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:58.113135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fa109-8995-4fab-b191-2a36c080f396 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.125114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15ab35-afe4-4d4f-b20a-f969b30cb7c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.166119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.167287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.167769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.168307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.169699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.171337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f80161-483a-4224-a712-59a4a34761a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.173456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/405] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:58.174781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.175280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.175750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.192659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:58.194851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:58.199587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.243406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fa109-8995-4fab-b191-2a36c080f396 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.267643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/406] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:58.269259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.287534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.300388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.301299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.301590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.301867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.303820 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.305603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.312070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 207/407] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:58.313141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.418333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b5a73-ca8c-4bc0-a236-0737acebbc77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.438351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39177319-bb17-4448-8972-943afea6eb70 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.462685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.464139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.464426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.464979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.466578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.468800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.471801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/408] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:58.472950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.480530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.481398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.481684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.481961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.485746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.497180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:58.508748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:58.509911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:58.511239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:58.514865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 208/409] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:58.515696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.576650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.614680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd1d8-c7ad-45fa-bb44-6ff06fd167a6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.645730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b86a-1390-4993-b25e-04ba88668020 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.660389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.661543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.661825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.662431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.665449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, group_id=ec5c9745c73a4ec787a7a9f03fdcf333, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.667571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.686874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/410] 10.222.0.45 () {58 vars in 1442 bytes} [Wed May 25 13:38:58 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:58.687699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.697129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.698099 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.698617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.698896 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.706160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5f8d2d683da441baa1480f203594842, group_id=ec5c9745c73a4ec787a7a9f03fdcf333, role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.707728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.732372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 209/411] 10.222.0.45 () {58 vars in 1442 bytes} [Wed May 25 13:38:58 2022] PUT /identity/v3/projects/f5f8d2d683da441baa1480f203594842/groups/ec5c9745c73a4ec787a7a9f03fdcf333/roles/798530c22da24d42b101d2f46d4939bb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:58.845750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78178107-07b1-4d34-95d1-7f5bc7887e7f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.906374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.907063 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.907350 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.907625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.908433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:58.908738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.910178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.916743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.925390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/412] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:58.931950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.001310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:59.054540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca0d8-4b9b-4806-8945-c106dc09c290 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.092519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.093387 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.093677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.093956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] PATH_INFO: `/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.095173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] RBAC: Authorizing `identity:get_group(group_id=ed8badf04c9c4a6192ca00a2ca41c1a8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:59.096704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09087d55-38a6-4994-95f8-8d9aafecc84c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:59.098561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 210/413] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8 => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:59.104390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.209797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744b8e-fdf1-4f3c-a32c-7eb0c6afcfbe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.233167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1718298-62df-45c1-b489-a9d71180892a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.233463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1718298-62df-45c1-b489-a9d71180892a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.233731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1718298-62df-45c1-b489-a9d71180892a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.235377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 211/414] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:59.240970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.241264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.241548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.246425 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.247476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.247764 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.248074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] PATH_INFO: `/v3/projects/c319a094752b41428929e5f2edeaa06e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.250111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] RBAC: Authorizing `identity:get_project(project_id=c319a094752b41428929e5f2edeaa06e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:59.251562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:59.253586 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/415] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/projects/c319a094752b41428929e5f2edeaa06e => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:59.260405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.263149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:59.264485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:59.359939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:59.362540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 212/416] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:59.398455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c9870-6f07-47fd-aeee-a271f118efd3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.402662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.464990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.465898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.466180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.466456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] PATH_INFO: `/v3/projects/c319a094752b41428929e5f2edeaa06e/groups/ed8badf04c9c4a6192ca00a2ca41c1a8/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.471119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c319a094752b41428929e5f2edeaa06e, group_id=ed8badf04c9c4a6192ca00a2ca41c1a8, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:59.473539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:59.498739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/417] 10.222.0.45 () {58 vars in 1442 bytes} [Wed May 25 13:38:59 2022] PUT /identity/v3/projects/c319a094752b41428929e5f2edeaa06e/groups/ed8badf04c9c4a6192ca00a2ca41c1a8/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:38:59.515025 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:59.569680 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aa9ea-3b59-4f40-a559-b0d19715cc96 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.609348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.610453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.610886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.611328 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.624600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1926b97d-ffe8-4ff3-9141-160f435b04b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:59.634162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:59.635310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:59.636334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:38:59.638756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 213/418] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:59.644390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.692874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523314-e77b-4e32-86da-df4d67539398 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.693982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523314-e77b-4e32-86da-df4d67539398 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.694462 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523314-e77b-4e32-86da-df4d67539398 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.696435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 214/419] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:59.702514 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.703100 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.703623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.735101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:59.736432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:59.787854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ad4e6-bfb2-4c9a-ae8f-44b763ec5aa2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.812647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 215/420] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:59.819956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:59.830690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.831368 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.831689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.831933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.832705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:59.833017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:59.834487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:59.840549 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.854835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/421] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:59.858942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.002091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d24f5-592f-4ad5-aeaa-4da2d0818287 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.010419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d04d-d099-4667-ace1-82379024c9cd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.028266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.058760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.059570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.059850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.060144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.075868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.076574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.078165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.078443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.087534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:00.104216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:00.105179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:39:00.105955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:00.118175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 216/422] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:00.125729 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.190137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.216647 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.218277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.294014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e01ba-f58f-4644-b185-6a4e7c746c53 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.302083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.374349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.377872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.378362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.378837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.379812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.380550 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.383061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.399757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 217/423] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:00.403345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/424] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:00.404837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.409482 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.569716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.587471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fa3c1-e522-42b0-8412-3295048e83e5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.592987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7341-2012-4e27-a9ae-ccb680a7f264 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.648610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.650298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.651647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.652447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.654695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.657363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.657790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.658205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.660424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.663503 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.669475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/425] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:00.675578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.717830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.775323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.778812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.847806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9f057-2afa-4261-8d2a-d9da7e61a422 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.915250 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.916377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.916801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.917216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.920252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.921864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.926095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/426] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:00.936906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.954640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 218/427] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:00.962171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.984334 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.053240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2c515-617b-42e1-aba5-d6592481259b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.074884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.114648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b409-d813-4a86-bde4-1dc9481428f4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.129215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.129886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.130168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.130472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.132006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=37bb98420135461aaed594a67f3f07f5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.132326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.133778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.148510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/428] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=798530c22da24d42b101d2f46d4939bb => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.155136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.156059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.156344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.156617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] PATH_INFO: `/v3/projects/37bb98420135461aaed594a67f3f07f5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.160815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] RBAC: Authorizing `identity:get_project(project_id=37bb98420135461aaed594a67f3f07f5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.163045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.165847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 219/429] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/projects/37bb98420135461aaed594a67f3f07f5 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:01.171621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.311445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.327694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6d7a1-93a3-4f6d-b7d0-28c91671d07f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.384924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.385776 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.386070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.386358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.387880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.389511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114e9bb-06ad-41eb-9784-f52946c88c12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.391400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/430] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.397736 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.531033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e01619-7c60-4c99-b21e-0bbc712ecd47 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.571809 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.572507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.572792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.573064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.573905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=37bb98420135461aaed594a67f3f07f5) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.574219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.575648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bccaa-22c1-40c3-93ac-2038109fa318 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.577636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.583442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225af0d-ce1a-438d-b15b-d7922f50a0d3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.584565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 220/431] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=37bb98420135461aaed594a67f3f07f5&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.584712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225af0d-ce1a-438d-b15b-d7922f50a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.584712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225af0d-ce1a-438d-b15b-d7922f50a0d3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.586606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/432] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:01.592094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.592613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.592688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.611391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:01.612628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:01.623322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06dd3-4c02-45e9-ad9c-bfa739e6b160 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.623650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06dd3-4c02-45e9-ad9c-bfa739e6b160 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.623866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06dd3-4c02-45e9-ad9c-bfa739e6b160 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.626493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/433] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:01.632778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.633071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.633327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.660830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:01.663729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:01.706025 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 221/434] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:01.731231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.764271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/435] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:01.787513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.871362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9eab0-07a3-4336-9a12-8057d25d14b0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.892886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.907418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.918269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d75fb-bb66-4010-b664-3e6b08ec16a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.921243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.922043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.922323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.922595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.937598 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:01.946062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:01.946974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:01.947762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 222/436] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:01.955165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.964908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.964908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.964908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.964908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.979653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:01.988227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:01.989272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:39:01.990061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:01.992545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/437] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:01.999796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.091018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4e6a0-e607-4471-bf21-bf7a5c9331ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.121811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfc03-10ac-4a99-91f9-dcf0468588df None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.131776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.132987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.133268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.133559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.134964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:02.135285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.136965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f76231-69ca-4b44-a59c-296e28fbb930 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.148968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 223/438] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.150336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533fb2e-b639-4bf3-b212-c174efa857fa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.150627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533fb2e-b639-4bf3-b212-c174efa857fa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.150903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533fb2e-b639-4bf3-b212-c174efa857fa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.152146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 224/439] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:02.154987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533fb2e-b639-4bf3-b212-c174efa857fa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.162785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.164140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.164426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.164698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.165458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:02.165767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.167175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911638-c04d-4059-81e7-87fc0eb18300 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.176674 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/440] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:02.177977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.178893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.179205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.197922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:02.199555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:02.201417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.220360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.279689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/441] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:02.280026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533fb2e-b639-4bf3-b212-c174efa857fa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.280566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.315110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.315782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.316118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.316401 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.380115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2094409-3d98-4890-86c1-34dde3d010f7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.386594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.388077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.415805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.416481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.416769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.417042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.424925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.436602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.485302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.486771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 225/442] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:02.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.487609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.584045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25107131-95c2-42c2-9e2b-44889c7a854f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.588594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/443] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:02.589434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.618825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.619729 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.620059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.620344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.628085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.631262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:02.639125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:02.640161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:02.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 226/444] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:02.641616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.660641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.688350 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8167f-d22c-4df5-a2bc-ec92f1b4185e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.722393 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.723247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.723533 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.723809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.725102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.726806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.728521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/445] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.729284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.749216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cae80-bbf6-4c03-a67a-b49bbecd7f37 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.783090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.783976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.784263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.784545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] PATH_INFO: `/v3/projects/f5f8d2d683da441baa1480f203594842` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.785735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] RBAC: Authorizing `identity:get_project(project_id=f5f8d2d683da441baa1480f203594842)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.787746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.789601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 227/446] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/projects/f5f8d2d683da441baa1480f203594842 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.790367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.831500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cfdfa-d1d9-4d12-91af-e7175961c7f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.839209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.867627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.868172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.868333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.868576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.868850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.869731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:02.870075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.871471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.879434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/447] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.880241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.893306 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c8018-54c3-4ce7-a790-210b6dfeac2d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.926907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.928327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.928792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.929065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.930228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.931650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.933284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 228/448] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.934034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.978211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e650-d8d8-4df9-b011-4068b62bc674 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.011322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.012193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.012487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.012766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] PATH_INFO: `/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.013931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] RBAC: Authorizing `identity:get_group(group_id=ec5c9745c73a4ec787a7a9f03fdcf333)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.015365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.017288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/449] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/groups/ec5c9745c73a4ec787a7a9f03fdcf333 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:03.018060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.038877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3408bd-632e-45f7-beae-6e942b8fb5b2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.052197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.073252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.073905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.074191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.074466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.091236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.118229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1c93-20e8-45f9-9eca-15e5b177fb03 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.144291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.145739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.150573 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.151218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.151495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.151765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.152616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=eda7e112a67a4f12993c9af8f1f3b587, scope.project.id=f5f8d2d683da441baa1480f203594842) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:03.152929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.154352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.160170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/450] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=eda7e112a67a4f12993c9af8f1f3b587 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:03.160936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.262748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 229/451] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:03.263966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.273709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.277956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a21d-be11-466c-9963-b67b16616f09 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.319335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.320038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.320739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.321027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.321306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.322257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] RBAC: Adding query filter params (group.id=ec5c9745c73a4ec787a7a9f03fdcf333, role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=f5f8d2d683da441baa1480f203594842) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:03.322606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.324244 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.330849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/452] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/role_assignments?group.id=ec5c9745c73a4ec787a7a9f03fdcf333&scope.project.id=f5f8d2d683da441baa1480f203594842&role.id=798530c22da24d42b101d2f46d4939bb => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:03.398354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112c99-4958-4be5-b522-db17c8d3c632 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.437966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.439101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.439380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.439653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] PATH_INFO: `/v3/projects/c319a094752b41428929e5f2edeaa06e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.440926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] RBAC: Authorizing `identity:get_project(project_id=c319a094752b41428929e5f2edeaa06e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.442378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.445419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 230/453] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/projects/c319a094752b41428929e5f2edeaa06e => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:03.451202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.566605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb5b6e-17f6-4354-a823-c529df1ee71f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.602523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.603384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.603671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.603973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] PATH_INFO: `/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] RBAC: Authorizing `identity:get_group(group_id=ed8badf04c9c4a6192ca00a2ca41c1a8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.606622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b478c46-9384-4df2-88e0-bd7aed6cf35b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.608280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/454] 10.222.0.45 () {56 vars in 1180 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/groups/ed8badf04c9c4a6192ca00a2ca41c1a8 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:03.614317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.716963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e45510-ae93-4036-9ef9-86cd36efb861 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.732199 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.752216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.752876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.753159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.753433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.754260 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] RBAC: Adding query filter params (group.id=ed8badf04c9c4a6192ca00a2ca41c1a8, role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=c319a094752b41428929e5f2edeaa06e) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:03.754573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.756037 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4aa831-4ce2-45ce-9ca5-9c45be2d992f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.761726 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 231/455] 10.222.0.45 () {56 vars in 1376 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/role_assignments?group.id=ed8badf04c9c4a6192ca00a2ca41c1a8&scope.project.id=c319a094752b41428929e5f2edeaa06e&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:06.540747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794666be-68c4-4ab8-a26b-cc4682e0ce9e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.541752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794666be-68c4-4ab8-a26b-cc4682e0ce9e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.542243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794666be-68c4-4ab8-a26b-cc4682e0ce9e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.544054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/456] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:06.549714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607ab6c-427f-4e8e-ae06-dca881fbda7d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.550533 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607ab6c-427f-4e8e-ae06-dca881fbda7d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.551120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607ab6c-427f-4e8e-ae06-dca881fbda7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.576887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5607ab6c-427f-4e8e-ae06-dca881fbda7d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:06.578434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5607ab6c-427f-4e8e-ae06-dca881fbda7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:06.680913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 232/457] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:06.716629 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794666be-68c4-4ab8-a26b-cc4682e0ce9e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.738188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4626e-1f06-4d28-8402-15a662fd7d01 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.738805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4626e-1f06-4d28-8402-15a662fd7d01 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.739284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4626e-1f06-4d28-8402-15a662fd7d01 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.741137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 233/458] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:06.750293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8393-e47d-44f6-b266-f5d650422a35 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.750928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8393-e47d-44f6-b266-f5d650422a35 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.751411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8393-e47d-44f6-b266-f5d650422a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.772290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.779218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a9de8393-e47d-44f6-b266-f5d650422a35 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:06.782411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a9de8393-e47d-44f6-b266-f5d650422a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:06.866351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 234/459] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:06.866716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794666be-68c4-4ab8-a26b-cc4682e0ce9e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.867942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20ff9-b109-44ae-9b46-536dbed2a339 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.876438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20ff9-b109-44ae-9b46-536dbed2a339 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.876871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e20ff9-b109-44ae-9b46-536dbed2a339 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.878714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 235/460] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:06.884865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.885433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.885869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.910535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.911369 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.911628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.911906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.913511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:06.914702 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:06.927147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.929579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.946604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:06.950914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:06.952736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:06.953721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:06.953721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:06.953721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:06.953721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:06.955370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/461] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:06.961562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.986272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 236/462] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:06.987366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.071011 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.096069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715d4dc-992f-4c1b-9415-83aa782a0f7b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.096307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b6c5-4977-4b1d-b87d-a2ffe56a836a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.132978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.133890 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.134314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.134802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.135991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.136602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.138655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.145199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.146127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.147060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.147433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.154869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 237/463] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.156214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.170517 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.172642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.192989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:07.195049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:07.196167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:07.197446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:07.197446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:07.197446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.197446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:07.199641 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/464] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:07.200673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.294610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de68bf1-b7df-411e-9c99-f50d51cbb45a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.308927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4776a9-7b41-4eb0-bab2-89696f413f30 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.335125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.335934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.336232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.336513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.347948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.350529 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.350880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.351257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.351540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.351815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.354935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.354935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7eaaf-53d5-49db-9407-92c7bb84aa56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.363969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:07.370358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:07.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:07.372333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:07.372333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:07.372333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.372333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:07.372688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 238/465] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:07.373489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.386861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/466] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:07.388597 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd41a4-4548-4823-9f1c-7387ca47b0d7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.389034 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd41a4-4548-4823-9f1c-7387ca47b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.389385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd41a4-4548-4823-9f1c-7387ca47b0d7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.395126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/467] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:07.395126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd41a4-4548-4823-9f1c-7387ca47b0d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.497333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.521177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a57d9-0b93-483c-9021-4281a6c6cc8f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.556141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd41a4-4548-4823-9f1c-7387ca47b0d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.565891 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.567444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.567971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.568522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.570406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.570986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.572895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020d6b8-1322-4931-9d37-b6ac97eae496 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.588307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 239/468] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.589968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.590258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.590537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.609568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.610243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.610535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.610814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.610909 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:07.611550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.611864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.612058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:07.613288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.636078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/469] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.636973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.698951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 240/470] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:07.700135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.704257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.764964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f460cf5-f592-4702-a43a-c0bc1fee3fd3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.779542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.812448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.813129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.813416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.813694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.814409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.814852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b92b3-f5eb-48cc-88eb-836f51b672ae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.815873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.843962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/471] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:07.843962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.862488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.863292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.863649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.864016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.864809 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.867104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1407fcc-48a8-4263-8526-4a76169c258f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.891845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 241/472] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:07.893523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf16e-9a68-4f97-b156-7d3ca24acb17 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.893953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf16e-9a68-4f97-b156-7d3ca24acb17 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.894294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cf16e-9a68-4f97-b156-7d3ca24acb17 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.896225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 242/473] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:07.903361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.903656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.903915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.924180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:07.926606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:08.011336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 243/474] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:08.034345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fb21b-dba4-4728-976d-b22e7b50354a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.035910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.051730 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.102853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.103984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.104444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.104898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.106513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.123307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.125243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:08.146046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:08.147021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:08.147880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:08.147880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:08.147880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:08.147880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:08.148511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/475] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:08.155598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.162979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644635d1-0a59-43d3-b1f3-23022fe011ab None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.209859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.211015 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.211305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.211586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.212412 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.214639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.224270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:39:08.224270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 13:39:08.224270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'd84e3caf59b24574a5e32066861b44d4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 13:39:08.224270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 13:39:08.249344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return ref.to_dict() May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:08.250253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:39:08.251173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:39:08.252397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:08.253271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:08.254078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'd84e3caf59b24574a5e32066861b44d4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:08.254866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 13:39:08.257438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.259368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 244/476] 10.222.0.45 () {60 vars in 1131 bytes} [Wed May 25 13:39:08 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:08.259368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.286245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9296-abd9-4432-b67a-55e4e53bbb41 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.324528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.325289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.325701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.326056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.326867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:08.327252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.328766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.342746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/477] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:08.348014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.371248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006dc8a-53ee-411d-89cd-4c7e3075bf58 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.392312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.403261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.410989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.411805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.412330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.412719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.423727 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:08.432972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:08.433871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:08.434846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 245/478] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:08.448973 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.518685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011157-570f-4492-8f9c-ca728ab4e804 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.555846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.556784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.557640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.558062 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.558934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.560583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2da8-b11d-4417-b3a4-10f2dd8b87af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.586199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/479] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:08 2022] POST /identity/v3/users => generated 249 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:08.651292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f770c3-ed51-4a26-91a1-e9d2ea65e5f5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.688339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.689003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.689283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.689553 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.690357 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:08.690669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.692070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12479e1-e480-4f54-8a91-e9e7a98cecf3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.702200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 246/480] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:10.322078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b850-cc2d-4e2c-9755-fb09f8084396 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.322590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b850-cc2d-4e2c-9755-fb09f8084396 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.322666 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b850-cc2d-4e2c-9755-fb09f8084396 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.324372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/481] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:10.329623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.330048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.330201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.349387 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:10.351002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:10.389878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:10.425796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 247/482] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:10.450038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b850-cc2d-4e2c-9755-fb09f8084396 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:10.560612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:10.599949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b850-cc2d-4e2c-9755-fb09f8084396 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:10.651440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:10.652545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.652976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.653397 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.668986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-529e3544-93bf-4e3e-b99a-9eddbeaef727 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:10.677151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:10.678831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:10.679830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:10.682815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/483] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:10.689057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:10.796560 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611abe-eb13-4596-889a-e6d1ff12b4b9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.796849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611abe-eb13-4596-889a-e6d1ff12b4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.797133 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611abe-eb13-4596-889a-e6d1ff12b4b9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.798503 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/484] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:10.810596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f4d03-2e4e-473a-a7e3-1176abf9804c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.811296 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f4d03-2e4e-473a-a7e3-1176abf9804c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.811911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f4d03-2e4e-473a-a7e3-1176abf9804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.830278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-521f4d03-2e4e-473a-a7e3-1176abf9804c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:10.831814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-521f4d03-2e4e-473a-a7e3-1176abf9804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:10.874330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ec8e2-e1dd-4aec-ba56-5a694dec6dd4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:10.912049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/485] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:10.912465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:10.913330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.913625 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.913906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.914626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515478-c615-4b74-8455-32a67cfc8a01 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.914728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:10.914962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:10.915411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515478-c615-4b74-8455-32a67cfc8a01 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.915884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515478-c615-4b74-8455-32a67cfc8a01 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.916381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:10.917952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/486] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:10.923758 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.924344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.924824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.929918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 248/487] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:10.935294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:10.950964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:10.952535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:10.973463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.015969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.031943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/488] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:11.031943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.082013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d57fe-0d98-4b2b-9d1e-0d38e498cbbb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.142702 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06324e86-f511-404e-943f-7f25ae3e309a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.144631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.145427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.145709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.145984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.164922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.167067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.184949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.186085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.186580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.187056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:11.187950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.188774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.189621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:11.189621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:11.189621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:11.189621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:11.194317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 249/489] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:11.195400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.203187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:11.211612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.212566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:11.213372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/490] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:11.214664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.291208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.308436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.329174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d1371-612a-44e9-af8a-142ebbf1fceb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.363298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e629cd-2f69-4eba-bf8b-1b97142c2783 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.396185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.398148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.398595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.399014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.421182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:11.426855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.428327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.428805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.429201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.430041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.433546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.439952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:11.444214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.445135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:11.445923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:11.448552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 250/491] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:11.449551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.457152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/492] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:11.458073 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.562841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be93d73-bb38-4dfc-8fbf-c0ea189796e8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.577313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfc74e-e760-44b2-8a58-162fef6795d5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.594470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.602671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.603345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.603632 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.603908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.604715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.605028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.606394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ea771-982d-4dfe-ba98-bf0485829cc4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.613531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.614190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.614480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.614711 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 251/493] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:11.614813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.615515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.615839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.616483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49270b-85e5-4565-98cd-90d0c38d3cd8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.616603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49270b-85e5-4565-98cd-90d0c38d3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.616839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49270b-85e5-4565-98cd-90d0c38d3cd8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.618167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 252/494] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:11.619134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49270b-85e5-4565-98cd-90d0c38d3cd8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.622137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf970f25-cecc-4583-8e62-bf25a54a1b40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.630825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.636000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/495] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:11.636000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41272d32-2bc2-479a-92ac-b39413ef62ad None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.636000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41272d32-2bc2-479a-92ac-b39413ef62ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.636000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41272d32-2bc2-479a-92ac-b39413ef62ad None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.636000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/496] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:11.636871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41272d32-2bc2-479a-92ac-b39413ef62ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.742086 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49270b-85e5-4565-98cd-90d0c38d3cd8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.748498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41272d32-2bc2-479a-92ac-b39413ef62ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.776468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.777251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.777536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.777814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.781448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.782235 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.782522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.782801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.785350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.787640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.793970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.795415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1d6903df-457d-4948-bba8-2c4a0fd1aa34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:11.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.804810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:11.805706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:11.808072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 253/497] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:11.809577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a205-c62a-4035-bdb5-54f09264927a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.809872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a205-c62a-4035-bdb5-54f09264927a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.810151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a205-c62a-4035-bdb5-54f09264927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:11.812785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.813696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.814515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:11.814515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:11.814515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:11.814515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:11.814515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/498] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:11.814984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.826739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a601a205-c62a-4035-bdb5-54f09264927a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:11.827956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a601a205-c62a-4035-bdb5-54f09264927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:11.847212 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.889870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.896503 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 254/499] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:11.897886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.898181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.898461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.914393 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:11.915562 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:11.932711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e284f9-e403-440b-82e2-3ca6a58cdaad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.966692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.967477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.967762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.968070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.979699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.981218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.982610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 255/500] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:11.983447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:11.998846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.999852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/501] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.000745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.082724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.092691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a91f5b-7d6c-4196-b728-235bf3ddbb1a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.102809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e6a97-090d-4a08-8466-df7aa40cd064 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.121344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.129769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.130431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.130869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.131160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.132037 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.132355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.133790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.136370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.137029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.137318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.137600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.138320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.138631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.140019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.152300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/502] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.153111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.170748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 256/503] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.172645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.257841 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b5be-98ad-4518-a770-61dac220013c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.272813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf6354-6389-46c8-9063-7e13b2b366c1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.291935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.292738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.293017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.293290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.303618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.305156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.306840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.310709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.311367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.311654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.311945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.312689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.312994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.314342 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:12.323887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:12.324766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/504] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.326496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.331908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 257/505] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.331908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.379205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.430818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b517b-59a6-42d6-9178-8f49acb7db96 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.433474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2abaea-e001-421e-8b48-9d9e47cd9851 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.464138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.464923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.465206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.465481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.466255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.467045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.467341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.467615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.476115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.477113 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:12.477616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:12.485278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:12.486193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:12.486968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/506] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.487640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:12.492311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:12.493217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 258/507] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.498916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.546978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.591431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc240b-bdce-4123-b263-8c63a5a174f5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.598790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c93a71-4dbb-494a-809f-f5fc6812703e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.606543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.626959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.627678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.628013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.628290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.634861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.635514 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.635798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.636109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.636832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.637136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.638502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.650389 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 259/508] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.651188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.699437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.700940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.749896 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6333c-d22e-459b-b27f-24ec7e9d13fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.761669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.785128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.785919 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.786203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.786483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.794159 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.795604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.807415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/509] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:12.808221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:12.811047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:12.812043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:12.812935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 260/510] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.813470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.841530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.907298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efcea9-1574-4789-b7cd-b0bf182ec754 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.913410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ec119-04d3-4b4a-b169-f556d0db2598 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.940913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.941578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.941865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.942140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.942892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.944282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.945609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.947480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.948168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.948454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.948732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.949566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.949877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.951284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.958166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/511] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.958166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.969139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 261/512] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.969814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.980186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.054563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.062664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06abeb6-1002-4dbc-b9e1-6225aa6bdb16 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.071818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724453be-633c-4f9a-aa82-80701cfd55d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.096198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.096920 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.097208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.097491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.098199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.099708 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.107018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.107806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.108114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.108396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.119689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.120217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/513] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:12 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:13.121046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.121165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:13.151457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:13.152480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:13.153301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:13.153301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:13.153301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:13.153301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:13.153301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 262/514] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:13.154022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.209878 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.244875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0457683-8fd0-41b2-8163-ac53ddab9d6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.276526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b91ad6-dff9-427a-b4f0-5ea2e23295d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.282997 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.283691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.284004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.284284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.285285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:13.285601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.288001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.314510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.315239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.315584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.316020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.319470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.328984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.329596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/515] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:13.332019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.332177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:13.352285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:13.353101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 263/516] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:13.361255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.454277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1287-093d-4af0-a3f6-28834cc9f30e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.469242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab12cc5-03d7-4214-bf4a-1ceaf087dd8e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.488340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.496554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.497207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.497494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.497772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.507161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.508122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.508589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.509041 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.509945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:13.510435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.512020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.525134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 264/517] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:13.526206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.575355 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.576825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.581627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.636691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05fb9-cb04-44dd-a76f-fb789c48e779 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.671731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.672417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.672701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.672976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.688344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/518] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:13.689526 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.720878 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.748938 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.750388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.794413 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eccd2-b66e-431e-b14b-418a775aec3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.800835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.831443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:13.832185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.832472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.832751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.833483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:13.834483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:13.836015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:13.853415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/519] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:13.853415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.862390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 265/520] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:13.863455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.953206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:13.965301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd0b22-d459-48a5-ae6a-c0252229213b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:13.968522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d46f7-daf3-4b7f-b7d2-5a18e3c6a8ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.001574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.002374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.002655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.002925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.005200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.006589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.006872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.007143 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.011671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.013187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.017798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.019299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.028943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:14.029951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.030984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:14.031840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/521] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.032332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:14.044473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.045320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.046289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:14.046289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:14.046289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:14.046289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:14.046289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 266/522] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.046554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.144945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8c71-0f3a-4a55-992a-ee2dfdffc5e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.155321 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64980ab-e459-449f-abe6-2e3c07a1af07 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.181765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.182554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.182831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.183100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.190099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.190545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.191331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.191567 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.191664 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.191969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.204492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.205922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:14.208197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.209673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:14.210710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:14.212184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/523] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.212184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.224461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:14.230110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.231026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.231875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:14.231875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:14.231875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:14.231875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:14.231875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 267/524] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.232212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.289179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.329775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c8b98-ce9d-488a-afbe-43c01d2b6024 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.345986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b351b34-9f33-4c4f-94b7-004047493a98 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.375631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.376379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.376598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.376874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.377659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.377968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.379415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.383554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.384275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.384567 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.384843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.385562 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.385871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.387227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.399609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/525] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.400780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.403972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 268/526] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.404743 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.475855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.510798 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023632d-a417-4700-87b7-7c0456c7ce0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.516388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010aa930-2bda-426e-be34-5fc0cf93ff82 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.530408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.548822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.549494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.549777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.550050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.550887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.551199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.553186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.556613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.557492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.557953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.558402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.559290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.560067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.561782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.572017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/527] 10.222.0.45 () {56 vars in 1139 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.573207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.574944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 269/528] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.575912 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.685806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8c9cf-850b-4c9e-b810-e95cb260a3c7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.690214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e111b4a-760b-44ab-b64e-94c2418c20f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.719144 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.727990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.728533 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.729136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.729328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.729614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.729964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.730066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.730606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.731618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.732336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.734300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.737456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.738890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.741206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 270/529] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.742216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:14.765760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.766673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.767609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:14.769838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/530] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.770951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.799284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.880440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2ce93-bc33-4808-a09a-3a554f48e82b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.921374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12040482-c281-47d1-8ef1-3232de03c126 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.922205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.923613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.924312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.924805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.967455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.968831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.969355 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.969821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.978888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.980780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:14.997398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.998291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:14.999121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:15.001753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/531] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:15.002729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.017531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.019344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.061741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.064770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.120864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde3da-1904-48af-8619-b45bc802c51c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.157624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 271/532] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:15.157624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.163750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.164712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.165189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.165655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.166650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:15.167162 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.168824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.187041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/533] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.188209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.280240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17361cb3-c3b9-4147-a1f7-659366be1591 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.309415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ef925-aa33-4eaf-b27c-a411a3c3f8e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.319421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.323783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.324479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.324770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.325051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.325852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:15.326164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.327791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.333259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.352201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.353351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.354434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.354936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.356078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 272/534] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.357213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.370289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.372298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:15.392913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:15.393820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:15.394654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:15.394654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:15.394654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:15.394654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:15.401478 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/535] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:15.402580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.473175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e64e-646d-489f-be92-218433ec42fb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.509952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.510895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.511358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.511813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.512878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=3e02e0b0eac04b51970fa87a0df61e64) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:15.513371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.514977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.516296 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b25755-822c-4e65-a53e-9d4c555230ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.522444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 273/536] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/role_assignments?user.id=3e02e0b0eac04b51970fa87a0df61e64&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.523778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.569411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.569995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.570277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.570554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.571417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=593fe462b2d947259086a4f98a043752) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:15.571736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.573285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f19651-4b50-4b72-ae3b-a557561333d2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.587964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/537] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/role_assignments?user.id=593fe462b2d947259086a4f98a043752&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.587964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66d00-41b8-42ba-ad36-6d536f6eab2d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.587964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66d00-41b8-42ba-ad36-6d536f6eab2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.587964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c66d00-41b8-42ba-ad36-6d536f6eab2d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.587964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/538] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:15.591362 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.591650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.591936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.627701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:15.629013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:15.640969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.674874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.732590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f3db5-ff6d-48b9-b5fe-e94b9ba6e054 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.768028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/539] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:15.791351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.792076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.792425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.793741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.795836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.801127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:15.801722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.803498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.818257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 274/540] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.828783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.995592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5a567-c777-4b07-8163-b301ce4b8be6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.007680 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441614a-17ff-487d-8f96-b80f7be71633 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.044331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.062917 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.063711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.064014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.064288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.069676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.070471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.070750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.071022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.085720 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:16.093881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:16.094808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:16.095589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:16.096767 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.097492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.101854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/541] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:16.105948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:16.118924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:16.120989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:16.123044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:16.125327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 275/542] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:16.132467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.134239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.273962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dfa2e-11c3-4793-9a36-a3a37a0694b3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.322261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dda9c-7d9b-45b0-a8d3-3a378ed80dc2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.348483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.349068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.349998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.350289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.351322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:16.351638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.353095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.374192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/543] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:16.376140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.388672 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.389344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.389632 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.389909 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.390717 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:16.391032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.392529 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.413474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 276/544] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:16.418286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.494018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.520313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.564103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddc9c0-b125-4ac6-ac95-8802d4ae14c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.600304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedbec2-427b-45e2-a311-060ff4c8411f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.624494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.625299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.625584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.625859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.643741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.645426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.667365 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.668040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.668328 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.668604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.669447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=eb026adfce894a72b97d750cdbb72efc) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:16.669764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.671208 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.677363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 277/545] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/role_assignments?user.id=eb026adfce894a72b97d750cdbb72efc&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:16.681015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:16.681836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:16.682656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:16.682656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:16.682656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:16.682656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:16.684423 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/546] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:16.699581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.852758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699be5b-a23d-483a-9ac9-a2af0c3a9be9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.855498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.902343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.903252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.903680 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.904132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.905030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:16.905481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.906996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.922051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 278/547] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:16.927793 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.038120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c1e8c-9c21-4de4-8339-6a2ba92e80dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.065794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.075381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.076215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.076500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.076936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.097835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.097835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:17.122227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:17.123877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:17.130742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:17.132197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/548] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:17.138343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.241908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac0331-1058-4f98-a26f-065b6f9efae5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.278269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.279258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.279731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.280209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.281220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:17.281709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.283317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aafaad-42d8-4477-be12-fb9e2aac717c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.301559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 279/549] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:17.308988 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.392297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a7dd-867b-4afc-a156-d39e77e6b533 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.392974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a7dd-867b-4afc-a156-d39e77e6b533 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.393454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52a7dd-867b-4afc-a156-d39e77e6b533 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.395499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 280/550] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:17.401830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.402407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.402882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.437056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:17.438638 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:17.451663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad42ef-96b7-461f-aaac-573e16e9975e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.467915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.494683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.495349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.495641 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.495943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.569726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.580447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 281/551] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:17.584312 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.586143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.614775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.690298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/552] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:17.696696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.735337 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb370dc-d25c-4ad3-b25d-9953f77d1a35 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.795126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.804167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.804996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.805268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.805534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.808622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb75e9-b8d0-4f93-a5e3-0202b1ed1fab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.839420 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:17.844520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.845306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.845591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.845864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:17.856444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:17.857505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:17.858334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:17.858986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.859076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 282/553] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:17.859657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.864882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:17.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:17.880060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:17.884023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:17.884023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:17.884023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:17.884023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:17.884023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/554] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:17.887378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.928323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.999887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f9d5-60f9-4af8-9481-ece6b7666d99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.035980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.036642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.037058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.037978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.038710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.039017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.040439 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.053525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/555] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:18.054127 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.059078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.148055 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99ae15-e3e3-4fad-a1e2-1e81db2f2f72 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.170190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c6383-4ca9-415e-a3bd-cca8e67a6134 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.210875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.211685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.211994 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.212277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.219481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.220969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.233215 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.233900 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.234183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.234457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.235216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.235525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.236920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:18.237745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:18.239092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:18.240515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/556] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:18.245950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.254264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 283/557] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:18.257175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.314936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.346334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.361177 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5afc3-8966-49a5-9e49-40748b9cf1a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.376532 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1e51e-349a-4305-84fe-84bd7de6b879 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.398006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.398752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.399090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.399346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.400231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.400568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.402046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fb5dd-9d43-4d75-91db-ff6635f69717 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.413475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.414268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.414559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.414829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/558] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:18.414931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.416104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaab73-108f-4ab5-a17c-625d57a13514 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.416395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaab73-108f-4ab5-a17c-625d57a13514 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.416670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaab73-108f-4ab5-a17c-625d57a13514 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.417941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/559] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:18.419004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d769dc-9e06-459d-8581-51fa693f34be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.419287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d769dc-9e06-459d-8581-51fa693f34be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.419802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d769dc-9e06-459d-8581-51fa693f34be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.421292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/560] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:18.422002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d769dc-9e06-459d-8581-51fa693f34be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.433255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.434715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-878d4d20-df10-4ace-9bf5-ab0675885f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:18.453082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:18.454126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:18.455040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:18.455040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:18.455040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:18.455040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:18.455040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 284/561] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:18.455850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005261a4-5a48-4585-9018-b37c86e68bda None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.456161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005261a4-5a48-4585-9018-b37c86e68bda None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.456441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005261a4-5a48-4585-9018-b37c86e68bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.474289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-005261a4-5a48-4585-9018-b37c86e68bda None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:18.475860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-005261a4-5a48-4585-9018-b37c86e68bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:18.534199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d769dc-9e06-459d-8581-51fa693f34be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.547615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 285/562] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:18.549073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.549364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.549645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.556668 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.565937 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:18.567126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:18.574909 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.575785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.576188 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.576566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.577623 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=644184b8dbe74a2e9704ee090c465b57) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.578033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.579665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.586101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/563] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/role_assignments?user.id=644184b8dbe74a2e9704ee090c465b57&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:18.589271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.630212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.653092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 286/564] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:18.654197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.745279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e269b-4a96-4700-82f5-ee2d5155c9c6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.785337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.786267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.786698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.787153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.788074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.788537 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.790115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.804624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fa8a9-7c20-42ab-9ce2-ca27481d44ac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.807398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/565] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:18.808446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.860831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.861645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.861934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.862213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.863404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.878209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:18.889431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.890576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:18.891495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 287/566] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:18.892206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.932375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b439b91-07ad-4a76-a268-89a998e29767 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.954773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.973477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.973820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.974243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.988508 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:18.998403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.999538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:19.000609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:19.007363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/567] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.007363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.029626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc6ebb-5335-4d52-8dd5-7ff2670658d6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.065673 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.066492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.066843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.067185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.077116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.079101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:19.096011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.097212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:19.098351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:19.104394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 288/568] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.104394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.136182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229658f9-bd1a-461c-aadc-bb7f0c169285 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.139436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.173359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.174677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.175554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.177480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.178321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.178751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.180290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.190311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/569] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.191273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.251878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed947c4-9379-4ef7-8004-77e7978a746a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.260286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.289014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.289686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.289982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.290257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.291064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.291380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.292794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.306364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 289/570] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.306364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.321080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5e6d-e8d0-45a0-984a-5a0dd4a5d880 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.366858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.367811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.368270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.368693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.370262 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.370745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.373170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.392313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/571] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.393142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.423553 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.424120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b0936-3e6e-4ec0-bdf7-753e79e94ac0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.460645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.461741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.462026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.462300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.473244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.475246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7847976b-c945-43ae-9958-de80b28a71ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:19.493035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.494117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.495292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:19.495292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:19.495292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.495292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:19.498825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 290/572] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.499981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993c5c-e171-4e87-86d6-60e22286402c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.500270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993c5c-e171-4e87-86d6-60e22286402c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.500543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993c5c-e171-4e87-86d6-60e22286402c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.502059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 291/573] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:19.502622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993c5c-e171-4e87-86d6-60e22286402c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.515511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656e107-5026-4fcd-93a0-385d74e822f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.529701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.551466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.553182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.553846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.554120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.564665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.566109 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:19.585928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.587229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/574] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.588714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.621331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993c5c-e171-4e87-86d6-60e22286402c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.655670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.656480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.657053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.657330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.668161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.668584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.670286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b0550722-d45f-46c5-8678-de2cce79234d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:19.688042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.688949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.689830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:19.689830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:19.689830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.689830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:19.689830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 292/575] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.690580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.690692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.690971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.699255 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc1240-7f9b-47cc-8309-fba5a02f393b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.709823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:19.711494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:19.735136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.735792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.736100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.736380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.737105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.737408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.738766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.756400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/576] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.756400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.790295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 293/577] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:19.792169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.795674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.871291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b8f1f-e8dd-4c36-ac03-58d44770598a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.902474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0ffd-8974-4b5a-9eda-2dd821de887b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.908488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.909174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.909455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.909722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.910445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.910748 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.912762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.915493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.926950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/578] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.928397 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.943567 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.944432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.944648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.945028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.945760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.946066 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.947436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.960292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 294/579] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.965309 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.032722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe77c-595f-4326-9722-73f0c42d2818 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.035133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.070680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.071473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.071759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.072071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.079315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.080478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074db8e5-8219-453d-a333-5623cd4e0493 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.080791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:20.096428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.097366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:20.098323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/580] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.100045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.123782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.124631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.124975 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.125321 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.136378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:20.144092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.145003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:20.145915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 295/581] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.147171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.168697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.218821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0329e-f9ed-483f-b3c0-33564aa50128 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.256530 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.257331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.257617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.257894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.259355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa35adb-7a06-4ab8-bb08-ca3339e6d90f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.265412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.266896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:20.282589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.283589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:20.284995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:20.285570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.288698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/582] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.289567 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.298477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.299275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.299564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.299844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.307868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.309354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:20.324253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.325163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:20.326063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 296/583] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.326545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.401701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177d1d-28b6-4597-9154-95e9bc73fa3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.419069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.438505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.439177 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.439461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.439744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.440610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.440925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.442377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.449213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206788e7-696a-424e-86f6-0fdeb39d7a53 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.459885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/584] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.460753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.490459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.491223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.491510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.491783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.492571 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.492879 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.494238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.502953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 297/585] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.503769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.552136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.573769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb54a5-def0-4755-b9af-a1b947e78c88 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.609743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.610402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.610677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.610953 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.611790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.612129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.612450 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00caf4-59ac-4218-92b8-b0c7fb8830d0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.613574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.632744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/586] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.633564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.658154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.658790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.659143 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.659299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.660149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.660466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.663718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.670473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.678954 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 298/587] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.680301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.750385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dfb9f-5f81-4910-9b9b-a21838d83320 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.790464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.791267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.791551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.791830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.801029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4b8f-ce6d-48e7-868b-d1a8f787ccc1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.803018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.804593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.805331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:20.823889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.824877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.825761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:20.825761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:20.825761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:20.825761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:20.825761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/588] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.826049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.846409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.847369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.847955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.848247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.860028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.861482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:20.879177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.880263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 299/589] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.882395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.917113 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.933379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad454142-c8dd-410d-b8e4-a1e3b4201d8c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.967656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.968476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.968760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.969035 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.979738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.981216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.982012 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaf7d9-102d-4981-961a-6088626322cd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:20.998683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.999622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/590] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:21.000800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.018793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.019697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.019989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.020273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.034935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=6086ccae239045e5b109e33cf47453a2, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.036405 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee3e13-7b12-48ce-b196-805d445b3ef9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.048754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 300/591] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:20 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:21.050023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7e85e-5e57-4cb3-88b1-df933845ee6b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.050313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7e85e-5e57-4cb3-88b1-df933845ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.050591 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7e85e-5e57-4cb3-88b1-df933845ee6b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.051867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 301/592] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:21.053107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7e85e-5e57-4cb3-88b1-df933845ee6b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.063094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.131984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197cb50-77b3-4b23-a40d-9f905f294bb4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.172502 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.173204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.173493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.173779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.174528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.174843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.176288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.176789 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7e85e-5e57-4cb3-88b1-df933845ee6b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.179848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.195417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/593] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.195417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.222237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.222915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.223201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.223477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.224292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.224631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.226323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dac05-9c9b-4e35-acf3-2713e5ddc6f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.241622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 302/594] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.243856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.243856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.243856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.266700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:21.268023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:21.317678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9899bd-a6d8-4542-a8dd-d602841705e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.332579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.342114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 303/595] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:21.342899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.360201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.361114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.361646 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.362021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.363093 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.363507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.366307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.382483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/596] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.384392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.449166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.456315 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093e021-452d-4484-85d1-a5f494f5f221 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.492897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.493701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.493998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.494270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.499392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7607-d612-4ac2-bd38-56d9ce456004 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.501240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.502695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:21.517521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:21.518526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:21.519471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 304/597] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:21.520014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.544692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.545660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.546045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.546416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.556184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.557754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:21.573168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:21.574326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:21.575463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:21.580901 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/598] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:21.581744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.588399 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.644750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3794d9-4d0f-4c22-a645-c96a208a5e87 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.679947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.680788 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.681071 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.681344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.684644 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36326cfb-14e9-47e4-842d-b5c320bd9526 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.692330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:21.700898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:21.701941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:21.702786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 305/599] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:21.704264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.704854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.726467 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.727257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.727544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.727818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.734638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.736129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:21.750031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:21.751098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/600] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:21.752026 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.824966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23913a7f-e6a2-407f-9636-00e6ab4aaff7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.846542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.864819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a8b2-2f98-46a5-ba40-0093410ca899 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.871645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.872954 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.873350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.873731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.875081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.875505 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.877776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.894651 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 306/601] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.895703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.905982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.906836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.907280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.907750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.908635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.908949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.910398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.933944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/602] 10.222.0.45 () {56 vars in 1139 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.934810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.969263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.997747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd9859-7eec-4c0b-baad-da2e515ded9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.031134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.031793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.032108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.032385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.033130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:22.033439 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.034818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.041066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbea3b0-be0e-42e9-b9c1-eb18567b0655 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.047242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 307/603] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:22.048103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.076979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.077710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.078148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.078539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.079435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:22.079859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.081892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.088630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.103613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/604] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:22.103613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.154786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a428699-783f-4b60-8d46-f9f9842ffae0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.195999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.197021 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.197392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.197733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/593fe462b2d947259086a4f98a043752/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.201590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.210367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab018d-4bea-4357-a47f-c6aa5db57c56 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.215545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=593fe462b2d947259086a4f98a043752, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.217021 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.244040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 308/605] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:22 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/593fe462b2d947259086a4f98a043752/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:22.244040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.247964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.248872 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.249223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.249566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.266198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.267717 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:22.297019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:22.298061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:22.298956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:22.298956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:22.298956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.298956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:22.301880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/606] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:22.303950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.375167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.397344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925542a4-3aeb-484c-a285-3924b41807e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.442225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19d0f-8ffd-41ac-9930-fc88d0eaf131 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.448025 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.448871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.449103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.449372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.462678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.464531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.481980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.482866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.483151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.483428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/3e02e0b0eac04b51970fa87a0df61e64/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:22.486424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:22.487277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:22.488298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:22.488298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:22.488298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.488298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:22.490730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 309/607] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:22.491995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.499686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=3e02e0b0eac04b51970fa87a0df61e64, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.501172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.504314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.536146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/608] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:22 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/3e02e0b0eac04b51970fa87a0df61e64/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:22.536146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.648876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389df64-46f3-4064-be19-1fd01db2c4b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.729483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70702475-6998-47d8-af48-b3819811df7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.730281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70702475-6998-47d8-af48-b3819811df7a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.730455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70702475-6998-47d8-af48-b3819811df7a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.730732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70702475-6998-47d8-af48-b3819811df7a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.731465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70702475-6998-47d8-af48-b3819811df7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:22.731769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70702475-6998-47d8-af48-b3819811df7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.733168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70702475-6998-47d8-af48-b3819811df7a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.749259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ee75f-714b-481d-9751-6e2d9cc7a119 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.755419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.768264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 310/609] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:22.773981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70702475-6998-47d8-af48-b3819811df7a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.791810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.792667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.793237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.793702 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.794549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:22.794964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.796610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.810151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/610] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:22.817511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.892297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.908746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70702475-6998-47d8-af48-b3819811df7a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.950184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.951016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.951301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.951578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.959585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.961212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:22.977276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:22.978203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:22.979301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 311/611] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:22.981825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a32157-abd3-48bc-9615-bb04c1a99433 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.985625 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.057531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.096666 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.097857 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.098593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.099051 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.112096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.113805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:23.129825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:23.131937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:23.132888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:23.133423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7ecbb-25d0-403a-b58b-3a39e1bc56b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.135117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/612] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:23.152115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.216327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.220610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.221177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.221595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.226861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.227439 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.229184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.259310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 312/613] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:23.265285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.311887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde7a79-a32b-47b9-8fcd-d0ae02eb498f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.330132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.372664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.373523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.373929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.374315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.375276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.375693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.377288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.415140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/614] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:23.420873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.434131 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1b4e-a404-4574-ad37-13859cb095fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.455153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.471056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.471950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.472238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.472508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/eb026adfce894a72b97d750cdbb72efc/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.488118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=eb026adfce894a72b97d750cdbb72efc, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.489690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9cbf-5595-4ba4-9433-5641fdec26eb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.507311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 313/615] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:23 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/eb026adfce894a72b97d750cdbb72efc/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:39:23.509282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4a79-fa66-4fd8-bbd1-4591efe2d07e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.509590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4a79-fa66-4fd8-bbd1-4591efe2d07e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.509893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4a79-fa66-4fd8-bbd1-4591efe2d07e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.512368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 314/616] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:23.518745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.520715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.521030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.565604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:23.567207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:23.569654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e65fd-8454-408a-ac98-7c1dafd25101 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.617819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.618630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.618907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.619178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.632155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.634217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:23.656660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:23.657628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:23.658513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:23.658513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:23.658513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.658513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:23.658513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/617] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:23.664359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.714583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.749272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 315/618] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:23.781019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.801932 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212604-e647-4ce8-befc-86e38bd4d847 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.846557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.847146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.847477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.847689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.848444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.848750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.857132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.883210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/619] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:23.889213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.998018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.016407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08153-268d-4e16-b689-3e4352032c7e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.022438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.026277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb8615-f4a7-4865-9635-25dd0ebd144b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.068135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.068135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.068135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.068135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.079849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.080670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.081156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.082207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.094004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:24.101367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.102889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:24.107256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:24.108181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:24.109129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:24.109800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 316/620] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:24.121810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:24.127835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:24.128712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:24.129570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/621] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:24.141050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.298287 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d21c8-3b88-48de-ad64-7b0f0384762b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.328284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afa230-46e6-42cd-8b2b-2f8996735d91 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.380300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.381320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.381707 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.381984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.382741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:24.383049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.384451 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.400404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.401070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.401352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.401628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.402423 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:24.402735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.404202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.421263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/622] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:24.422172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 317/623] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:24.423348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.429029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.433930 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.480335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.606536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6605b2-f783-4cfe-94aa-c3d581fed7f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.617586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee3e48-b472-4a1c-9077-5d831cea98aa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.670094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.670910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.671192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.671466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/644184b8dbe74a2e9704ee090c465b57/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.678322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.679128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.679410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.679685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.693974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.695690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.703045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=644184b8dbe74a2e9704ee090c465b57, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.704572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:24.715018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:24.715953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:24.716864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:24.716864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:24.716864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:24.716864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:24.716864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 318/624] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:24.721968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.752319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/625] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:24 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/644184b8dbe74a2e9704ee090c465b57/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:24.815586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.868079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da885858-ddba-4bc0-94d6-d5bbf68277c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.918204 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.918896 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.919185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.919464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.920364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:24.920628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.922097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa359b-6b6a-4d54-b1fa-37e254594a55 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.941723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 319/626] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:24.947001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.015634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a410d-9a0c-4ffc-b9f5-717b0f8164de None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.016334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a410d-9a0c-4ffc-b9f5-717b0f8164de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.016817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a410d-9a0c-4ffc-b9f5-717b0f8164de None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.018518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 320/627] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:25.020911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.026668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.027331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.027682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.046076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:25.047654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:25.079313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67298-e0b3-46c3-89ac-800fa5ecd505 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.121909 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.122808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.123186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.123534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.132455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.134207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:25.151464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:25.152521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:25.153549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/628] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:25.157985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.181385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 321/629] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:25.233570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.304443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7992a-3e4e-48bd-8126-32692a29b77d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.309491 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.328692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.350088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87708d-d9ad-4002-95e7-f2fa87ebf6f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.350205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.351114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.351360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.351639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.355538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:25.355861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.357366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848200e-f79f-4c7f-89c1-c086154094d2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.377182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/630] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:25.378735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fe0b-49b5-4cd1-94aa-46622fb64473 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.379036 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fe0b-49b5-4cd1-94aa-46622fb64473 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.379320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59fe0b-49b5-4cd1-94aa-46622fb64473 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.385373 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/631] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:25.386304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59fe0b-49b5-4cd1-94aa-46622fb64473 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.394045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.394832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.395113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.395382 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.406482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0dc93f24-f4d7-4f98-be09-bc1d98b639cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:25.415042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:25.417353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 322/632] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:25.419374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.421347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.421347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.436076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:25.437185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:25.498257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59fe0b-49b5-4cd1-94aa-46622fb64473 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.507850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 323/633] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:25.507850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.560559 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.561321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.561619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.561903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.564214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.609836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082c730-a037-4f13-b417-f5694184cb0e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.621898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.644338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.645130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.645801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.646645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.647038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.647512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.647790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:25.648188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.649533 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.658217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 324/634] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:25.659113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.756408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/635] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:25.757627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.772077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4cecc-8c95-4872-bd8f-ba53d6f7fbd5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.789981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.809815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.810612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.810899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.811170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.822130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:25.829686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:25.830610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:25.831445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 325/636] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:25.832185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.847497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.882890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f106e-53af-43ef-9d80-07610f34c116 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.930219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.931293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.931581 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.931853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.946443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.947934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.952944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394fdd-a50a-4f2e-97e6-91a92a0ba21a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:25.966604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:25.967514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:25.968444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:25.968444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:25.968444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:25.968444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:25.968444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/637] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:25.969524 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.994669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.995462 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.995781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.996073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.007900 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.009319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-77ecd3a3-e7d9-4412-a557-d6d0f03d9d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:26.026816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:26.027772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 326/638] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:26.029390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf6769-5032-426a-a81e-c89ff7f1e132 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.029739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf6769-5032-426a-a81e-c89ff7f1e132 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.029739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf6769-5032-426a-a81e-c89ff7f1e132 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.031151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 327/639] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:26.035596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf6769-5032-426a-a81e-c89ff7f1e132 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.071292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.084267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4070cf-46d5-478b-ad8c-77d276480b28 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.112782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.122962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.123636 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.123915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.124230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.125000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:26.125314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.126698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.135855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/640] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:26.136713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.167029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf6769-5032-426a-a81e-c89ff7f1e132 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.208099 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.209333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.209479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.209670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.210392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:26.210703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.212093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb9910-98df-4305-930c-1cc0a9af0695 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.226222 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 328/641] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:26.226750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.227257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.227749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.245555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:26.246331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8114e0b-e516-4446-b4b6-07d4ee09b943 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.246737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:26.282070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.282957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.283238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.283509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.284282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:26.284594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.285978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.298037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/642] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:26.298849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.331790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 329/643] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:26.333031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.362628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.365422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.423080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbdd3c-bd2b-4b65-a354-c7a809df4312 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.453703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6a55-67e1-49bf-a780-0599fe412507 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.461028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.462007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.462120 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.462367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.472963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.474508 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.491626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.492455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.492743 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.493023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:26.493234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:26.494123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:26.495020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:26.495020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:26.495020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:26.495020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:26.495020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/644] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:26.495305 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.504443 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.506085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:26.521843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:26.522942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 330/645] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:26.524054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.607427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.609974 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63d679-8199-40cd-88d4-72824d8040da None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.615433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.638929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b54b1-c557-4b55-931e-f6a1b0f3498a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.650304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.651291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.651760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.652224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.660561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.662343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.674335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.675125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.675412 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.675681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:26.677579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:26.678749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:26.679812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/646] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:26.680389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.693479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:26.701539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:26.702461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:26.703300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 331/647] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:26.704018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.796550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8241d-be78-49fb-93bf-f90303c2bee8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.814542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ddb98-506a-461c-a715-4253f079fcd0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.834505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.835172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.835456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.835732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.836490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:26.836802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.838189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.854706 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/648] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:26.854706 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.856426 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:26.857101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.857389 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.857718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.858509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:26.858818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:26.860269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:26.869905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.878549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:26.880288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 332/649] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:26.886571 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:26.965509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b443d9-762f-4c25-a6a0-e532fe7d6170 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:26.995594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd948d-3d61-4c0b-9350-7c3e3796be81 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.007280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.008247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.008749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.009207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.010194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.010687 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.012345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.026014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/650] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.027077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.036781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.037457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.037742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.038020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.038774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.039085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.040482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.048683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 333/651] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.049483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.114283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.117004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.139192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d870c-6887-495f-88ee-030b987c9733 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.161444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1a060-6731-4aea-bce9-54d287456ef3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.172769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.173563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.173849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.174128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.182794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.184351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.195968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.196608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.196888 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.197157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:27.198813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.199801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:27.200791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/652] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.201421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.266528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.267993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f19dc5-464a-4ddf-a47d-3501ed241cf4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.299946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc47380-053e-427f-a82d-1ba5c572621e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.324279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.333394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.334966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.335249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.335520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.336378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.336686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.338114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.343791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/653] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.344594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.348576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.376837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 334/654] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:27.378316 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fad2-1d0a-4732-aae6-e3553bd78905 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.378618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fad2-1d0a-4732-aae6-e3553bd78905 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.378904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fad2-1d0a-4732-aae6-e3553bd78905 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.380476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 335/655] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:27.381499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fad2-1d0a-4732-aae6-e3553bd78905 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.474398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c6d12-d14b-42e6-a70a-ef4904ca2704 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.505043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fad2-1d0a-4732-aae6-e3553bd78905 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.516276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.517092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.517700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.517995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.530749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.532065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.547448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.548154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.548441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.548713 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.549557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.549872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.551299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d88c7-cd2d-43c7-ac0c-8ca199e9e399 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:27.552178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.553770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.555601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:27.555601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:27.555601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.555601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:27.558162 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/656] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.559863 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.574821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 336/657] 10.222.0.45 () {56 vars in 1137 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.576143 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.576427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.576704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.594545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:27.595726 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:27.608862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.630310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.669487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 337/658] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:27.670701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.688343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3105d-5bec-44f6-81ec-e4f98750aa2b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.727315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428209c3-b827-4325-a594-356326b52d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.728340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428209c3-b827-4325-a594-356326b52d20 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.728480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428209c3-b827-4325-a594-356326b52d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.728680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428209c3-b827-4325-a594-356326b52d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.739694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428209c3-b827-4325-a594-356326b52d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.741413 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428209c3-b827-4325-a594-356326b52d20 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-428209c3-b827-4325-a594-356326b52d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:27.759451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.760435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.761321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:27.761321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:27.761321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.761321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:27.761321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/659] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.762167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428209c3-b827-4325-a594-356326b52d20 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.784181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace3f1f-947e-4efa-9ccc-3a74f4df4db1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.823167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.824032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.824441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.824722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.825447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.825754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.827342 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.839961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 338/660] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.840811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.853976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.870775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428209c3-b827-4325-a594-356326b52d20 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.879682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.906359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.906927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.907216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.907498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.959399 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43b6f-34df-444e-ad28-2a0808cb12b4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.980579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.982599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.995549 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.996384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.996670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.996953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.008010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:28.015681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.016659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:28.017500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 339/661] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:28.018184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.089290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.102367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/662] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:28.102367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.112959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.131594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871b688-1e81-464e-be5d-54c69f87a46e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.171982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.172575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.172861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.173141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.173870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.174181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.175551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.188442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 340/663] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.189279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.220241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36dd5c-fe66-4a22-ba52-9b21503b621f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.260468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.261270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.261553 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.261839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.269717 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.271380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:28.287691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.289055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/664] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:28.289984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.297766 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03052cdc-7feb-492d-9e14-a23977417ec7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.330048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.340175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.340911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.341176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.341453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.342201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.342510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.344188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.362320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 341/665] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.362320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.368930 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.405033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e2da0-77fd-4061-90d6-4c84d85bb921 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.441686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.442792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.443663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.443866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.454175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.455634 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:28.473500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.474989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.480907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:28.480907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:28.480907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.480907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:28.481130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a7f98-d2ce-4704-b6fc-14bdb903e942 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.481206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/666] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:28.481206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.519114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.519978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.520261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.520959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.529271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.530755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:28.545883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.546778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:28.547683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 342/667] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:28.548220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.595236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb428498-7343-48c7-9381-b52f9e4eb1f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.598298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.617380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.634118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.634803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.635084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.635356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.636237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.636556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.638005 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.660111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/668] 10.222.0.45 () {56 vars in 1139 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.660296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2fd41-5096-4f0d-abeb-96db14b3831a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.660393 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.695600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.696431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.696720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.696999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.708333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.710942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:28.729291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.730165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.731042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:28.731042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:28.731042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.731042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:28.735176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 343/669] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:28.736046 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.763548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33056efc-0949-417e-9e53-f6601a1ee2a6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.797673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.798337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.798620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.798897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.799614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.799964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.801327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.819621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/670] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.819621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.849436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.853873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258778c-a3c6-4319-9a8a-998be1866bd5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.868326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.893498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.894150 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.894430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.894699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.895477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.895784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.897831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.916191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 344/671] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.916191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.938930 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9a508-4b5a-4bfe-a75f-157530eda104 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.976551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.977418 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.977718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.977996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.022610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562154a-a850-44d2-a8c8-292ad8356926 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.051306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.053421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.061631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.062299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.062584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.062863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.063592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.063902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.065288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.077813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 345/672] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.078669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.097581 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.107374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.169472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/673] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:29.171037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.196003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444c2ca-7a64-4fd9-b42f-2d6dc7ec671d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.231531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.232369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.232655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.232941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.240498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.242083 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:29.256810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:29.257922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:29.258848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 346/674] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:29.259335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.294795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafa5ab-2e60-467d-9571-9278f209161f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.332314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.333431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.334659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.334994 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.335844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=3e02e0b0eac04b51970fa87a0df61e64) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.336187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.337648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.352096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/675] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/role_assignments?user.id=3e02e0b0eac04b51970fa87a0df61e64&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.352096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.352934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.353053 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.381431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d6485-e67f-4431-926e-c18c9364fc08 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.420789 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.421977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.422303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.422655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.432521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.434151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:29.451395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:29.452524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:29.453458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 347/676] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:29.454055 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.508951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e1809-d27c-4a4d-97a1-c18ebe1c5c7d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.569529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.570485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.570928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.571207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.580383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79c64d-cc02-4cd2-9a0b-4acb67942453 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.586540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.588743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:29.609405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:29.610344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/677] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:29.612150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.627282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.628019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.628260 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.628607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.629413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.629720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.631717 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.641839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.647094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 348/678] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.648214 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.679098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.733509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb117df-b7fc-4232-8c9b-201807c84ce0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.754953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6b63b-b4d9-4c26-9611-96be11630393 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.770777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.772135 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.772424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.772821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.773777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.774090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.775537 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b755-3c9a-43e5-915b-c3f345a7eff4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.790185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.790861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.791147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.791424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.792196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.792507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.793868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.795693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/679] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.797773 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69014a3-99d0-4ad4-8c80-d38dc228d00d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.798238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69014a3-99d0-4ad4-8c80-d38dc228d00d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.798681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69014a3-99d0-4ad4-8c80-d38dc228d00d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.800356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/680] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:29.801833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69014a3-99d0-4ad4-8c80-d38dc228d00d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.806731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 349/681] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.807572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:29.875293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.913404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69014a3-99d0-4ad4-8c80-d38dc228d00d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.921927 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6deb-bce6-4a56-a643-cc98571a711f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.927889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.950254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.950921 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.951209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.951488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.952357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=593fe462b2d947259086a4f98a043752) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.952676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.954116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd958555-619b-4c72-87a7-b96ef1d473a8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.960201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/682] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/role_assignments?user.id=593fe462b2d947259086a4f98a043752&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:29.961386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.961676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.961956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.971737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.972767 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.973358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.973640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.985479 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:29.987030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:30.056562 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.057966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.093430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/683] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:30.093430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.143844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.200317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 350/684] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:30.201021 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.245677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e91db-d70a-4718-a700-54d84e2ac9a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.256765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.299956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.301164 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.301164 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.301337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.310747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.312272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:30.332412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:30.333372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:30.334371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/685] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:30.334866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.349623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0c112-cb8a-4b60-b406-bbd96b599ff8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.391035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.392232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.392685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.393205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.406128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:30.414961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:30.415894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:30.416889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:30.419326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 351/686] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:30.424950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.450139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.452800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca38-5e43-4d0e-9c80-50006daa697b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.491301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.492289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.492519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.492801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.504626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.506917 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:30.526055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:30.527165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:30.528327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:30.528327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:30.528327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:30.528327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:30.531952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/687] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:30.532888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.542645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a82199-8570-41d8-91cd-72a0ab89f630 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.554574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.585327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.586257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.586691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.587171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.588280 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:30.588743 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.590447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.610106 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 352/688] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:30.611359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.649038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b60d-9e7b-44ba-9aab-2b28ba7dece1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.686856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.687951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.687951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.688218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.688977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:30.689288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.690667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.699993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.701250 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/689] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:30.702292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.731675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd8879-4bb6-4371-add2-f23c1ad674b0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.770497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.771418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.771709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.772014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.772859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:30.773172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.774660 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.791543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 353/690] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:30.792594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.794083 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.815016 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d077c-3402-4ef7-abfd-32191ae5c34c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.860038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.860038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.860038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.860038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.860426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=eb026adfce894a72b97d750cdbb72efc) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:30.860426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.860552 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:30.876398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/691] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/role_assignments?user.id=eb026adfce894a72b97d750cdbb72efc&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:30.877413 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:30.930607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ef67-ab13-4944-bc5c-daa65ac56e4c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:30.973019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:30.973844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.974878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.975171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.987331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:30.990563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:30.993154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:31.015139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:31.016228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:31.017138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:31.017138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:31.017138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:31.017138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:31.017138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 354/692] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:31.025413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.111252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd355a-b1ce-479e-a82b-fa43ba060fac None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.166158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.166962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.167251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.167531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.177126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.178864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:31.195321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:31.197452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:31.199588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/693] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:31.200776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761928-92c0-4db0-99dd-a6f00e7ca6a5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.203277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.208605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.252824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.253494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.253784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.254065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.254976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:31.257050 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.258455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.273949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 355/694] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:31.279341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.355241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.360027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d12fe-deeb-4e97-9900-088f69623e0c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.401072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.401765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.402057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.402395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.403241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:31.403565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.405267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.415177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5899c-7664-4342-a06b-ae4140ecbb7e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.419339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/695] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:31.431889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.472773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.473954 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.476434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.476917 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.487525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.489243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:31.505956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:31.507447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:31.508408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:31.511010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 356/696] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:31.517235 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.541628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.619264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4329727-e625-46cd-9299-83af15ba40e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.681900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.682475 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.682759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.683036 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.683876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=644184b8dbe74a2e9704ee090c465b57) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:31.684219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.685777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.693928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfe49-0f27-47bc-987c-30e41ebd0127 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.702539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/697] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/role_assignments?user.id=644184b8dbe74a2e9704ee090c465b57&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:31.707361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.733859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.734536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.734825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.735104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.736050 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:31.736368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.737870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cbf8a-9364-42ef-a364-550260f93dbb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.759308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 357/698] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:31.764984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.777510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281a33-fd46-4281-a5e5-7cb71c7dfa80 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.777828 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281a33-fd46-4281-a5e5-7cb71c7dfa80 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.778077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281a33-fd46-4281-a5e5-7cb71c7dfa80 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.779771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 358/699] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:31.785769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.786110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.786395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.807512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:31.809123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:32.036315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6217bb-446c-4e3f-b19a-d5ad29974f84 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.047284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 359/700] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:32.072092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:32.081729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.082035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.082429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.082718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.083129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.167500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.169028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.216595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:32.220300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcde38-c607-404c-9a64-d42e0e9fa8e1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.266126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.267052 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.267431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.267763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.268527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.269977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:32.289300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:32.290288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:32.291244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 360/701] 10.222.0.45 () {56 vars in 1129 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:32.297020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.317064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/702] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:32.323596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.361890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:32.439827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2d45-352e-4689-83a6-6098529d5dc5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.461510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d68c3-558a-4521-8bea-dd4b7d559ff5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.477576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.478292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.478679 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.479055 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.479874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:32.480482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.482023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.488845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 361/703] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:32.492888 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.545734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.546456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.546741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.547018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.556335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:32.577830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.579780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1c30b9c8-d69c-4e3c-9ff9-9602def467ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:32.604195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:32.605163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:32.606037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:32.606037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:32.606037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:32.606037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:32.606037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/704] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:32.607025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c48-1314-40eb-8b26-f7f2a60edbe9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.607309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c48-1314-40eb-8b26-f7f2a60edbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.607581 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c48-1314-40eb-8b26-f7f2a60edbe9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.613524 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/705] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:32.614483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d2c48-1314-40eb-8b26-f7f2a60edbe9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.641724 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b15e7-abfc-470f-99bd-8e4b43826304 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.679914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.680883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.681170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.681527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.682878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:32.683196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.684854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a6768-98e9-43d4-902e-a78f51ec1bdc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.693737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 362/706] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:32.695103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.695404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.695678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.716439 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:32.717815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:32.718982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:32.776584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d2c48-1314-40eb-8b26-f7f2a60edbe9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:32.814324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 363/707] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:32.834596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:32.835285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:32.835570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:32.835848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:32.836734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:32.837051 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:32.838580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:32.856130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.858237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/708] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:32.876058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:32.897318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.014233 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575e30-fb31-44de-af20-b6f2311738f5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.037500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b888ace-93c2-4e55-8f2e-9d82127b2341 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.079139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.079859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.080175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.080443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.081647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.083111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.083111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.083111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.097836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.099562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.107565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.108180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.109045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:33.132048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:33.133018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:33.133895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:33.147392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:33.148388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:33.150304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:33.150304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:33.150304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:33.150304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:33.150823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/709] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:33.150823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:33.159994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 364/710] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:33.166130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:33.316148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.347360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71f878-8158-4ff7-aed9-e7e61e12545b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.385858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a38be-214d-4c50-b390-61a69cfd490a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.388986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.389864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.390343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.390764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.391688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:33.392185 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.394621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.408435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 365/711] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:33.413600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:33.460006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.460957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.461497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.461933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.462938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:33.463409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.465090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.479448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/712] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:33.485618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:33.620686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438027-8ac9-462e-b402-e0b49f6dbbc9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.656362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.656955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.657339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.657714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.658547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.660117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b776eda-bbfc-4098-9598-b0f5645f89f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.671444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.675950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.687250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 366/713] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:39:33 2022] POST /identity/v3/users => generated 254 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:33.763155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03e85f-77bc-4021-9453-49c82216361b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.796365 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7affdd-59d3-416b-a811-01c4735ece20 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.796998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7affdd-59d3-416b-a811-01c4735ece20 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.797517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7affdd-59d3-416b-a811-01c4735ece20 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.799349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 367/714] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:33.808741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.809372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.809851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.820122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.820790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.821071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.821346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.822191 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:33.822505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.823957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.830054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:33.831560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:33.838143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/715] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:33.942167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 368/716] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:33.965990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.086726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84529-7b83-41f9-a189-45019a8f57bb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.114143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.127078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.128263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.128795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.129285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.130154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.132443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:34.147505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:34.148590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:34.149440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:34.152311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/717] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:34.157849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.307658 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b44da-cfe5-40cb-a6af-858e01b6df40 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.329111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.366656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390eba83-4325-4060-be85-df9396868d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.367366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eba83-4325-4060-be85-df9396868d58 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.367650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eba83-4325-4060-be85-df9396868d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.367958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390eba83-4325-4060-be85-df9396868d58 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.368692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eba83-4325-4060-be85-df9396868d58 None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:34.369003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eba83-4325-4060-be85-df9396868d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.370418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390eba83-4325-4060-be85-df9396868d58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.375841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 369/718] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:34.381374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.533715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2ef18-0e12-43cc-b408-f73e40d64e99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.584129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.584696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.584986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.585265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.585987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:34.586298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.587689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7163a-d7ba-48b8-83b0-b6dc200d4283 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.594107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/719] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:34.695782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775edb8-a6b4-44ea-bb57-961d3de3977b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.696133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775edb8-a6b4-44ea-bb57-961d3de3977b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.696378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775edb8-a6b4-44ea-bb57-961d3de3977b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.697980 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 370/720] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:34.703353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.703637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.703934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.730877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:34.732106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:34.808801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.831223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/721] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:34.859000 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775edb8-a6b4-44ea-bb57-961d3de3977b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.977457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775edb8-a6b4-44ea-bb57-961d3de3977b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:35.019468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:35.020306 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.020590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.020866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.037175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:35.044785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:35.045738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:35.046622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 371/722] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:35.051911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.229824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1b9a-220e-4490-88ab-c629014e068c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:35.275658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:35.276357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.276638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.276922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.277661 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:35.277966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:35.279336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:35.289038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/723] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:35.294375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.354186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:35.425607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e725b8-e202-4dc7-a227-106992b826bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:35.477652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:35.478444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.478725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.478999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.497999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:35.499360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c65c152f-4500-4475-a138-9c54a20a3f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:35.523802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:35.524754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:35.525730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:35.525730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:35.525730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:35.525730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:35.525730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 372/724] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:35.530746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.588929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:35.625039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255669a3-3c64-4d5a-8c3f-054be087be9f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.625348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255669a3-3c64-4d5a-8c3f-054be087be9f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.625604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255669a3-3c64-4d5a-8c3f-054be087be9f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.626974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 373/725] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:35.633485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.633771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.634053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.656594 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:35.657824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:35.687140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3b46-e6a4-48c2-ac1a-c8816076f96b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:35.746858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:35.747528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.747809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.748167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.748842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:35.749151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:35.750529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:35.774686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/726] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:35.785086 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.800754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 374/727] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:39:35.831266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.860106 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:35.894345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79bd2-15aa-4105-b1c2-149944b05e48 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:35.897811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:35.936536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:35.937670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:35.938134 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:35.939285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:35.951534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:35.954208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:35.971419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:35.972437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:35.973366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/728] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:35.980831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:35.997557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc38a9-e378-44c3-ab75-50ace98eb768 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:36.037116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:36.038158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.038806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.040354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.041314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:36.043588 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:36.069082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 375/729] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:39:35 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:36.159436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee8f72-daa6-4199-84e6-86c1735bb632 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:36.218344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:36.219519 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.220087 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.220577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.221682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:36.222228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:36.224275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3034d-c21b-44c4-a635-d1896d4ba3cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:36.255138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/730] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:36.261155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:36.364120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:36.382800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d9a46-d83a-47ec-ae6a-e7799b164563 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:36.419448 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:36.420142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.420424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.420697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.451442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cef1e-9ce2-4474-82d0-8cc41aefbaa2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.451751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cef1e-9ce2-4474-82d0-8cc41aefbaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.452077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cef1e-9ce2-4474-82d0-8cc41aefbaa2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.453652 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/731] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:36.458795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.459081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.459361 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.488183 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:36.489367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:36.502034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:36.503869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:36.504342 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:36.564071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/732] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:36.590226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:36.652289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:36.680014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 376/733] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:39:36.681398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:36.789616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da84bf-3dea-49a4-8deb-c2fd214f9032 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:36.848209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8023-5672-48b8-91b9-52032c763e45 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:36.869677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:36.870923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.871401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.871868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.894613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:36.894613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:36.894613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:36.894613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:36.901630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:36.903530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:36.908936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:36.928263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:36.929253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:36.930078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:36.930712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:36.931483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:36.933130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:36.933130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:36.933130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:36.933130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:36.933130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 377/734] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:36.933404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/735] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:36.935410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:36.936820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:36.964290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.070753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4930f-7d80-4c3e-9f82-b1285f5c06b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.071418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.110013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.110693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.110980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.111259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.112054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:37.112374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.114057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.124516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/736] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.129815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.209871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e2ec6-ce96-4c72-9c75-ca38e9d1a4bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.248316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.248518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.249032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.249612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.249897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.250790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:37.251109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.253152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82bac-5772-4b0a-bd50-aba6a937afa9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.262462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cc083-22a3-4805-8be9-abbd1b1c773c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.270497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 378/737] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.272096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5344a7e-502b-469d-b5d7-a69d87f0262d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.272398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5344a7e-502b-469d-b5d7-a69d87f0262d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.272684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5344a7e-502b-469d-b5d7-a69d87f0262d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.278205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 379/738] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:37.278979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5344a7e-502b-469d-b5d7-a69d87f0262d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.305554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.306975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.307265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.307543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.320779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.323178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2682feb9-76fa-46a7-9aec-45eda986b0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:37.342002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:37.343382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:37.344317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:37.344317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:37.344317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:37.344317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:37.344317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/739] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:37.348456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.348456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.348456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.374603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:37.376621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:37.397196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.402694 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5344a7e-502b-469d-b5d7-a69d87f0262d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.446195 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.446988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.447273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.447547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.454360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.460213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/740] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:37.460367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.460451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-997d8bcc-f99d-4c75-9835-06dd792cb1e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:37.478297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:37.479425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:37.480561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:37.481618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 380/741] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:37.482880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bc0c-31ce-4bc7-8a21-81a352cba359 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.483172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bc0c-31ce-4bc7-8a21-81a352cba359 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.483454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bc0c-31ce-4bc7-8a21-81a352cba359 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.485141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 381/742] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:37.485863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727bc0c-31ce-4bc7-8a21-81a352cba359 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.518481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.571840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d22bf-913c-4fb1-b79b-49660b5eb324 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.602196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727bc0c-31ce-4bc7-8a21-81a352cba359 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.607875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.608578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.608862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.609254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.609983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:37.610294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.611817 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1efb2-ae00-486c-9e8c-9935e40e73c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.629409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/743] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.629409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.629409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.629409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.641466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.642261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.642574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.642852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.644486 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:37.646540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:37.654241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:37.662060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:37.662981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:37.663803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 382/744] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:37.664938 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.686951 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.724065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/745] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:39:37.724837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.775552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96903-f55d-49c1-bc61-7c682850466a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.778897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.811674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.812363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.812645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.813318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.814121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:37.814431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.816527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.829391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 383/746] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.835842 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.838375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217832-7461-4d04-9651-5fbead0174ab None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.875496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.876321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.876609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.876884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.884411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.885914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:37.901219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:37.902270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/747] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:37.903794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.925356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.950568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485be5b-499b-4bf4-9ac9-ace99977c5b9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.989995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.990587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.990872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.991148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.992063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:37.992375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.993739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.007453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 384/748] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:38.007646 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4ed67-f004-40ee-80dc-bcf2acd7999d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.007793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.007889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.043154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.043812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.044122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.044403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.045118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.046485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.071952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/749] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:39:37 2022] POST /identity/v3/services => generated 240 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:38.071952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.140868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b247e3-a66c-4826-a4ca-5c361fcf72ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.192621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.194017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.194486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.194846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.195806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=644184b8dbe74a2e9704ee090c465b57) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:38.196238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.197781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.201229 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.202320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013e669-b58c-486d-ba21-008f2cb67647 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.208347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 385/750] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/role_assignments?user.id=644184b8dbe74a2e9704ee090c465b57&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:38.209952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.254161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.255152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.256110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.256669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.257656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:38.257993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.259505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.282862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/751] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:38.288536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.314011 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.391065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d681e-0afe-4fa8-a9f1-26ca1922b45c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.425647 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1095dda-5b79-4d86-a8a9-8dd5c61dbcfc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.443535 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.445477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.446521 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.447162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.464014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.466677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.474227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.475095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.475379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.475897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.491562 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:38.491754 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:38.492614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:38.493477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:38.493477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:38.493477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:38.493477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:38.493705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.493777 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 386/752] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:38.498870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:38.517131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:38.518056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:38.518928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:38.518928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:38.518928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:38.518928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:38.518928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/753] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:38.525596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.566502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.628349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.632825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd3945-3bfc-452c-a7d1-5e8ee6031863 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.634411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aba1b-0b21-455f-a2e2-afbe1d3944b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.669737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.670408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.670585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.670739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.670956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.671690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:38.671806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.672029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.672409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.672846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.673411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.673882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:38.674343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.675852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.686007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/754] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:38.694654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.694821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 387/755] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:38.696462 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.825874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf678a-2132-47dd-830c-63ec29042576 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.828107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72c813-76cf-4c4b-acce-12d9a7612c3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:38.841823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.867230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.868103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.868456 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.868796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.874320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:38.876854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.878417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.882960 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:38.884007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.884446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.884875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.892590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:38.894238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b70c18bd-ab9a-445e-a753-4b609aa2e5a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:38.896785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:38.898031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:38.899201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:38.899953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 388/756] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:38.905140 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c150-8379-4005-9e06-0d60422a3d27 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.905455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c150-8379-4005-9e06-0d60422a3d27 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.905747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c150-8379-4005-9e06-0d60422a3d27 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.907278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 389/757] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:38.914153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:38.914487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:38.914746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:38.923292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:38.924240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:38.925138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:38.928771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/758] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:38.929866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:38.938626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:38.939793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:39.024411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 390/759] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:39.025812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.061862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf75f0-e362-4f4b-bebb-0a9d7b739ce3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.109961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.111203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.111947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.112410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.114153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:39.114736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.117092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.141912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/760] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:39:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:39.143160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.165794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c8ee1-afaf-4887-b103-10ba107d71dd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.167798 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.207146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.207807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.208131 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.208936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.210384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:39.210699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.212268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.216082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.231972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 391/761] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:39.232876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.280418 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aef7f-7948-4926-bb03-70c3511dd5a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.323165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.324672 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.325347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.325884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.326693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.328835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.351266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11795a6-f48a-46b2-9758-5be2e4e1c7da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:39.353200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:39.354318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:39.355379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:39.358866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/762] 10.222.0.45 () {56 vars in 1129 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:39.359902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.396782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.397569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.397977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.398351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.430038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.475295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.477231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.491366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b3c66-8517-4bb6-964c-a62dc8820217 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.507340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.533004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.534234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.534682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.535113 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.538198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=6086ccae239045e5b109e33cf47453a2, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.539932 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.559123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/763] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:39 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:39.560478 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.672160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 392/764] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:39.679640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.732217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496195b3-640b-4b2b-871f-0092bc68db0b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.798988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.802825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.803888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.804197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.804520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.805245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:39.805558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.806946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.815614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/765] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:39.821810 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.846512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bf3f-2f79-4fe8-98a2-764a58a1d694 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:39.875980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:39.901386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:39.902535 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.903018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.903466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.920053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:39.922545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:39.958113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:39.960082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:39.961189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:39.961189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:39.961189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:39.961189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:39.965916 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 393/766] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:39.972366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:39.994181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221e6f-9a00-4183-af5c-57f2fa1a8cad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.058067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.059181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.060772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.061275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.062276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:40.063583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.067948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.075544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/767] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.081511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.161014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9db9-46cc-4b1d-be95-8740deeefdb4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.185392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.217193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.217863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.218149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.218429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.219169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:40.219478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.220873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.234614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 394/768] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.241065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.276570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f2d2-73ca-4a1c-a00d-3096d1df9483 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.280238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.358095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.358842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.359132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.359416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.360339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] RBAC: Adding query filter params (interface=public, service_id=a4ab2d9b4b01417caaeca87011b818eb, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:40.360711 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.362108 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.370820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/769] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/endpoints?service_id=a4ab2d9b4b01417caaeca87011b818eb&interface=public®ion_id=RegionOne => generated 194 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.376129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.425440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59fc5-4a10-40b3-9721-e8c0910b9e9a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.463814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.464610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.465359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.465634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.473123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.475494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:40.495019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:40.496234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:40.497341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:40.498609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 395/770] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:40.507600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.580929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.625003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f25b-db9c-4ff2-b1f5-496b911aecf3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.685156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.685841 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.686128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.686408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.687107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.688543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9f489-0cf1-4c56-b7cd-8fa7d3758f00 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.690772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06306-aee0-4abd-9b6d-026e01608645 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.694981 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/771] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/services => generated 772 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.704597 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447090b3-353c-4e29-a005-33f379a65a40 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.704887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447090b3-353c-4e29-a005-33f379a65a40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.705172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447090b3-353c-4e29-a005-33f379a65a40 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.706928 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/772] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:40.712693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae4f14-b54f-4efa-93b9-4e6fa7753375 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.712991 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae4f14-b54f-4efa-93b9-4e6fa7753375 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.713261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae4f14-b54f-4efa-93b9-4e6fa7753375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.736656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.737974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.738258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.738599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.739419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:40.740497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.742319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.749050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cbae4f14-b54f-4efa-93b9-4e6fa7753375 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:40.750275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cbae4f14-b54f-4efa-93b9-4e6fa7753375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:40.760822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 396/773] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.773470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.808622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.860577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/774] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:40.862399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e561-2bc2-4866-aabf-0b0094be0ce7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.862908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e561-2bc2-4866-aabf-0b0094be0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.863464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02e561-2bc2-4866-aabf-0b0094be0ce7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.865704 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/775] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:40.872212 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.872816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.873306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.891905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:40.893534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:40.924307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.930994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b95d1-582d-4cda-868f-7f031dcbb9b6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.972722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.973527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.973883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.974926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.976172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=316eec64cd5f4184a0b97c946df81a54) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:40.976565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.978127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.984794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 397/776] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/role_assignments?user.id=316eec64cd5f4184a0b97c946df81a54&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.985806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.993433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/777] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:41.016497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.130887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:41.155146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb014f-9896-4741-b135-f70944613cb3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.184786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e35bf9-19c9-44b6-9f71-760a9572e690 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.226161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.227993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.228627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.230302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.247207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:41.251218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.252308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.252796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.253292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] PATH_INFO: `/v3/services/network` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.254156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.256539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:41.260531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:41.261453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:41.262265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/778] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:41.268816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:41.272617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:41.273646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:41.274455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:41.277239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 398/779] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:41.284625 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.332278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:41.456807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da982394-ac4e-437c-87cb-2db4297bbe0b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.474397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ad27-aa22-42ff-ad64-f6186560c9ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.512462 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.513078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.513308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.513580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.514356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:41.514663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.516050 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.522423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 399/780] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:41.530399 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.546252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.546931 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.547218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.547497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.548290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:41.548604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.549997 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.567907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/781] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:41.572998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.598460 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:41.674490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:41.691137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b779a-3583-4334-9bdf-93c207b8c725 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.728495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.729541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.730031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.730529 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.731457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:41.732002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.733573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.739393 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 400/782] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:41.748806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.760511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd0687-630e-4349-a8fa-88abf1dc7fd7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.833591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.834900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.835181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.835457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.857731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.859197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:41.882713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:41.883632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:41.885157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:41.885157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:41.885157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:41.885157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:41.885157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/783] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:41.891943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:41.929122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d1d70-7c0e-46f7-b34b-18ada844e4a1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.966118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.967203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.967722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.968264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.969315 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] RBAC: Adding query filter params (interface=public, service_id=bbe933955b0146d4b939acec63f12f37, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:41.969867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.971494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.983053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 401/784] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/endpoints?service_id=bbe933955b0146d4b939acec63f12f37&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:41.988178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.016291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.079159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.110162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa9173-19db-4a29-ba8b-887f948ed61b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.150728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.151408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.151700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.152011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.153100 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:42.154503 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c3ec-6691-4cad-9dc4-79408640f92f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:42.160752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 402/785] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:42.181071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b653ab-2213-472f-b177-4c368c361cfe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.218834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.219521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.219807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.220163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.220920 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:42.221222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:42.222657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:42.236663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3195fb-6593-400b-8686-4d3fe644934e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.237112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3195fb-6593-400b-8686-4d3fe644934e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.237228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/786] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:42.237509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3195fb-6593-400b-8686-4d3fe644934e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.239160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 403/787] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:42.243599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.244700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.244994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.245276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.263747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:42.265098 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:42.386547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75e49-8ac4-4619-b729-184851e3b4f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.401704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 404/788] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:42.424377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.435589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.436600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.436983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.437359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.445329 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.462945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:42.465500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.465814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:42.486924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:42.488194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:42.489182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:42.489966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/789] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:42.498314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.623775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7571b0-d47c-446b-83c4-a9f5b4e93e5f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.646277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cf83a-15c0-48df-82cc-b44f09a6f9b2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.670349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.671027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.671311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.671594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.673374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:42.673698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:42.675142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:42.701779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/790] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:42.708212 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.711041 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.714174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.714458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.714731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.735787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:42.745047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:42.745986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:42.747790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 405/791] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:42.755706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:42.810461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.863471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:42.905095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc066-1a7c-4319-bda4-aa1d9ebc1244 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.941982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:42.942649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:42.943592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:42.943884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:42.944698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:42.945013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:42.946458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:42.954045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bedd3-fbbd-4ad2-bb49-cc60cd8e149c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:42.955785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 406/792] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:42.961738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.028651 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.029462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.029749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.030024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.043187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.043187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:43.070122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:43.071068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:43.071993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:43.071993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:43.071993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:43.071993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:43.071993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/793] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:43.084951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.126017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de595d-7519-412c-b2a6-14ef2f66e776 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.161373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.177250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.178075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.178356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.178627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.200866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.202333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:43.230654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:43.231602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:43.232546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:43.232546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:43.232546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:43.232546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:43.241377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 407/794] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:43.246671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.255086 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.265416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbb6a-d96c-4324-a992-d306c72113c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.307403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.309414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.309968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.310426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.311343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:43.311831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.313782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.329427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/795] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:43.340335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.423857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaf2f3-d080-4195-8e1a-53f17d612628 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.451226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83475cd3-903b-48fd-b423-fc660757bec7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.461934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.462741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.463031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.463308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.464095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:43.464410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.466105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d592e1-add5-4125-a96d-65c17afde1e4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.482231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 408/796] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:43.483884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742ef2-3c2f-4cf2-ab87-13a782d53066 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.484201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742ef2-3c2f-4cf2-ab87-13a782d53066 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.484482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742ef2-3c2f-4cf2-ab87-13a782d53066 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.485965 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 409/797] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:43.488670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.489536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.490044 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.490279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.491394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742ef2-3c2f-4cf2-ab87-13a782d53066 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.500390 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.502964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-139d1954-a704-49ea-af9d-ef9c645a38f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:43.526357 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:43.527294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:43.528203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:43.529520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/798] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:43.530635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.530923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.531200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.548074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:43.549264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:43.549882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.561611 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.620826 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742ef2-3c2f-4cf2-ab87-13a782d53066 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.628233 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/799] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:43.629260 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.663077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.664829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.665334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.665809 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.674429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.676181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:43.694810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:43.695813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:43.696786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 410/800] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:43.697907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.758848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca2ab-56a2-416a-8b28-5e6eef0fd4e7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.796278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.796949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.797236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.797514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.798307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:43.798622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.800089 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a2b95-0dcd-451b-baf2-9967ee3faf0f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.808217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922a331-2d73-4fa6-8490-b89ddd437630 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.809910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.817043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/801] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:43.818631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577ace-4640-4688-8d55-6e2a15c39430 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.818930 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577ace-4640-4688-8d55-6e2a15c39430 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.819214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577ace-4640-4688-8d55-6e2a15c39430 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.820568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/802] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:43.821248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47577ace-4640-4688-8d55-6e2a15c39430 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.826447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.853707 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.854512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.854797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.855064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.855713 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.857097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:43.872032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:43.873169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:39:43.874181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:43.875350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 411/803] 10.222.0.45 () {56 vars in 1129 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:43.875907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.939399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47577ace-4640-4688-8d55-6e2a15c39430 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.976781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.977444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.977729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.977866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edceeaf-ff1e-424e-89c1-3eebf36679dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.978042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.978849 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:43.979165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.980642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49513ca0-6c6a-462f-89d3-fa9e26629f20 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.999975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/804] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:43.999975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.999975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.999975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.016979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:44.017319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.018200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:44.018302 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.018527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.018802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/644184b8dbe74a2e9704ee090c465b57/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.022602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=644184b8dbe74a2e9704ee090c465b57, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.024072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.051955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 412/805] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:43 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/644184b8dbe74a2e9704ee090c465b57/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:44.051955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.083509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.093120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.100356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/806] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:44.102076 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.196534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3522ab8-f3e8-4a83-8b30-82e6d10ac6ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.238787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.240609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.241116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.241873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.242982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:44.243504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.245140 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.253168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 413/807] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:44.254298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca405-6c85-423d-97c4-60b0f6de3483 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.254539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.296986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.297648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.297966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.298210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.370996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.382692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.384342 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.385585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.399523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bc45c-b65a-4837-8013-09306e1d83a7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.444178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.445152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.445540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.445917 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.456757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:44.465164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:44.466278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:44.467091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:44.472000 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 414/808] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:44.472867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.514105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/809] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:44.515498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.606236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf22f-8af2-4bc6-ac92-6d431f46133d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.628270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.645077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4b66c-b35c-441d-b1aa-3660d6db8e95 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.649870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.650760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.651225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.651682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.652695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:44.653184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.655269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.664077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 415/810] 10.222.0.45 () {56 vars in 1124 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:44.665591 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.687406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.691497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.692374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.692633 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.693660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.694485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:44.694862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.696344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.710508 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/811] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:44.711421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.797597 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3db54-bba9-4d7e-95cf-a60163f7ff5a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.820843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841ac8-7fae-4b13-9568-046bb453e996 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:44.833369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.834650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.834931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.835244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.846972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.848454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.857379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:44.858039 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.858328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.858605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.859353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:44.860761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07550873-ad6b-4a21-bcc5-0e92f001cf40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:44.868960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:44.870115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 416/812] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:44.872551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.891051 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/813] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:39:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:44.892229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26d1d9-f355-42f1-a79e-b0f7e3a15f46 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:44.892528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26d1d9-f355-42f1-a79e-b0f7e3a15f46 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:44.892874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26d1d9-f355-42f1-a79e-b0f7e3a15f46 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:44.894330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/814] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:44.899346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26d1d9-f355-42f1-a79e-b0f7e3a15f46 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:44.915912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:44.981734 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.035636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed37d-3dc7-4e4f-b713-03e818450b95 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.061629 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26d1d9-f355-42f1-a79e-b0f7e3a15f46 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.079508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.080346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.080633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.080911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.098109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.099850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.113855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.115283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.115283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.115947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.116682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:45.117755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.118384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f02a9d37-56e6-43ef-a7e2-de1d469b84c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:45.121157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:45.122036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:45.122931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:45.122931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:45.122931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:45.122931 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:45.125069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 417/815] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:45.126578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.126869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.127149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.132933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/816] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:45.134278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.152339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:45.153396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:45.211809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.253496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 418/817] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:45.254800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.269066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6f5fc-9878-497a-8b31-5b0652c177f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.316731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.317410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.317740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.318557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.319833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:45.320176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.322114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.329223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.339855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/818] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:45.340585 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.403209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad9900-ee02-4ebf-8029-8b6f95d5e828 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.447630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.448736 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.449034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.449313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.458644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.460107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.460679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e867fb4-a680-4efe-aca5-609db775afd9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:45.478395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:45.479383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:45.480325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:45.484400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 419/819] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:45.485281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.495733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.506783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.507940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.508440 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.508966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.509912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.511566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:45.525525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:45.526535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:39:45.527484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:45.530382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/820] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:45.531366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.616902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8286a-f5e3-439a-8d4a-dacaa6d6f720 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.628499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.648008 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b64c1-b662-4008-b5aa-c476ac4f3d62 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.662711 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.663950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.664239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.664794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.674449 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.676240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.685956 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.686897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.687410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.687898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.688924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:45.689457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.691090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:45.694851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:45.695787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 420/821] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:45.697347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.708887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/822] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:45.710038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.769141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:45.824607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290d2ad-b8d2-42e6-ade3-e0326976f99b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.831857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1b4cf-e2c6-491c-8ab4-62abb6124e75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:45.871149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.871811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.872207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.872553 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.873508 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:45.873828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.875376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.876436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:45.877435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:45.877933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:45.878464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:45.879385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:45.879913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:45.881716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:45.887545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 421/823] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:45.888589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.901175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/824] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:45.902411 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:45.930522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.015047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c08e3-389d-47ad-9ffd-c8ffa6e3461c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.024139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a283a4-c7f1-456d-9cd6-1ae50cba2c8b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.034086 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.057957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.059100 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.059643 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.060191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.061357 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:46.062069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.063848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.064309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.065075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.065458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.065836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.066923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:46.067333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.069757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.076280 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 422/826] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:46.077753 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.079961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/826] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:39:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:46.080802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.211829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955d58b-8aa4-4954-8703-0d3dff5370a1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.261236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.262240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.264246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.264669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.266437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.268902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25da12-9c3a-4f70-ad94-74c104933786 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.292923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.305921 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/827] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:39:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:46.307661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a21a7-9212-44cc-8922-058a3fe28717 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.368668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.369690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.370132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.370546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.374474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1951d-55dd-4f6f-b6ea-773d2034a62e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.374771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1951d-55dd-4f6f-b6ea-773d2034a62e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.375056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1951d-55dd-4f6f-b6ea-773d2034a62e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.376789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/828] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:46.382842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.383168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.383409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.387692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.389656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:46.411170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:46.413039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:46.413976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:46.413976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:46.413976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:46.413976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:46.416141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 423/829] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:46.417843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:46.419054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:46.422380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.524755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/830] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:46.549220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.557984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91107bb3-a41d-4d72-8e4b-803aeb55fdd7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.604307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.605117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.605402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.605733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.606490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:46.606930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.609001 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.627497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 424/831] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:46.634111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.638728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.678961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f61b-7848-4f0f-ab5c-afc900fc868f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.716164 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.716967 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.717253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.717528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.729441 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:46.737324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:46.738458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:46.739490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/832] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:46.746239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.752350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d2197-9ced-4df2-a6fa-3a64afba76d2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.774304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.804054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.804054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.804054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.804054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.814276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.815870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:46.835715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:46.836744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:46.837752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:46.840933 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 425/833] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:46.846725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.884183 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08637514-6ce3-4576-ab49-8c4508da4c7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.918850 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.936358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.937042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.937327 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.937599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.938360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:46.938666 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.940084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.949963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/834] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:46.955103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.968170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fa29f-7ff5-4ec6-8b7f-f6719576f189 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.017570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.018936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.019609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.020043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.021445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:47.022350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.024447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.041536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 426/835] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:47.046970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.053423 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.075642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76abf75-bb8c-4460-abb2-fe9c2f1e3844 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.121370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.122311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.122599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.122902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.138648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.140300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:47.165429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:47.166417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:47.167302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:47.167302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:47.167302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:47.167302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:47.171804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/836] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:47.172847 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.179627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465032a-ba5f-4635-a9b7-72dea1f442a5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.216098 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.230835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.231735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.232040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.232307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/316eec64cd5f4184a0b97c946df81a54/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.253560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=316eec64cd5f4184a0b97c946df81a54, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.255795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ea467-9f06-40eb-9732-cd784bc60f57 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.280112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 427/837] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:39:47 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/316eec64cd5f4184a0b97c946df81a54/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:39:47.306694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e3ed-faaf-4d33-ba86-4e3ecff181a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.341284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.350319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.350547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40b9b-850f-4bde-8b87-859ac1e8c112 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.350547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40b9b-850f-4bde-8b87-859ac1e8c112 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.350691 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40b9b-850f-4bde-8b87-859ac1e8c112 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.352367 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 428/838] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:47.352543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.353267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.353267 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.357770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:47.357770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.357770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.359318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.359672 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.359983 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.375688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/839] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:47.381513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.382388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:47.383812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:47.481303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 429/840] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:47.504661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.524121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa667-5e25-4f7c-93a8-6678598ee2af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.564033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.564859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.565149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.565432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.574223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.575698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:47.591712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:47.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:47.593618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/841] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:47.598780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.631165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e70e9-c2c0-4400-96c6-bf68ce21aee6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.633894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.664748 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.674425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.675236 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.675525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.675805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.676755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.679010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:47.695149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:47.696234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:47.697076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:47.699994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 430/842] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:47.702356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.731654 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d74e1-2e0b-46e1-b2d9-6096613b1aff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.768736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.769338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.769625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.769907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.770805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:47.771126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.772605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.795695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/843] 10.222.0.45 () {56 vars in 1135 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:47.796676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.841404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca92d-7a2d-48cd-93d4-dc9c2ae1c8b9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.889808 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.890508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.890796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.891074 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:47.891789 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:47.892144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:47.893504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:47.903498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 431/844] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:47.904380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:47.917191 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5acf69-5ce6-4a0a-81be-c35c74ace876 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:47.929388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.945996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:47.956870 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:47.957735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:47.958170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:47.958691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.026088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927fe70-2351-4a97-90c3-9a78d52c27e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.038714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.040807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.063474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.064414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.064878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.065261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.066155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:48.066568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.068869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.076192 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 432/845] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:48.169274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/846] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:48.177689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.284987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:48.294008 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56871ca0-59a0-4ddd-a739-a1040c426ae8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.330974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.331980 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.332274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.332551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.344720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.346167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:48.367123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:48.368077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:48.368947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:48.368947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:48.368947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:48.368947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:48.370898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 433/847] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:48.377015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.389496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:48.483014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92488659-566c-4cf3-8027-9a59f29d2efe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.522911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.523593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.523874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.524178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.524920 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:48.525229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.526629 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5cfbf-cbd5-48b9-a08c-7355009a4b95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.539485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/848] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:48.545117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.658654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75431d2-18fc-482e-b59c-17c302a6b947 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.690595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a821-3101-47a4-91b2-f79540a234e7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.690954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a821-3101-47a4-91b2-f79540a234e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.691161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41a821-3101-47a4-91b2-f79540a234e7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.693329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/849] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:48.694132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.694935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.695221 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.695490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.698287 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.698579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.698860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.703171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.705668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.715673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:48.716905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:48.722065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:48.722959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:48.723822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:48.728084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 434/850] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:48.733731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.745919 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:48.795242 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:48.801094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/851] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:48.822600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.861180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23129f-f017-43ad-8781-415f2a8883c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.907534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.908224 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.908511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.908790 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.909580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:48.910038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.911447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.925057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 435/852] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:48.933623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:48.935910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651ac3e-4b45-4e90-8c91-d8377e261d76 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:48.974062 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:48.975328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:48.975681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:48.976325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:48.977057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:48.978518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:48.997977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:48.998922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:48.999743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:49.002922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/853] 10.222.0.45 () {56 vars in 1132 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:49.008531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.017033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.063255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745c42e-2d08-4662-80e1-5569e59a88e6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.088050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.107863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.108567 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.108848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.109118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.109955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=644184b8dbe74a2e9704ee090c465b57) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:49.110264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:49.112070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:49.124125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 436/854] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:48 2022] GET /identity/v3/role_assignments?user.id=644184b8dbe74a2e9704ee090c465b57&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:49.140668 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1036-1ae9-4553-837c-e2a1f95bdf14 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.179707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.180777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.181288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.181787 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.182705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:49.183216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:49.184846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d227f-5c00-4469-a3b7-78a06862d23e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:49.190839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/855] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:49.197793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.313676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a714788-4e64-42c4-b4b3-47d47c57d181 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.349476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.362017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.362017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.362294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.363555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.363555 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:49.363732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:49.365779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a02fb0-02ea-400f-8064-25f3ea4f5361 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:49.371276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 437/856] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:49.696088 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c1672-e0b4-4b40-8701-d08170883c3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.696467 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c1672-e0b4-4b40-8701-d08170883c3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.696663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c1672-e0b4-4b40-8701-d08170883c3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.698283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/857] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:49.703727 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.704066 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.704350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.722846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:49.724175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:49.799428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 438/858] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:49.822941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c1672-e0b4-4b40-8701-d08170883c3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.932483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c1672-e0b4-4b40-8701-d08170883c3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.960755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.970774 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.971873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.972396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.972866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.985344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:49.993795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:49.994761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:39:49.995603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:49.998459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/859] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:50.004117 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.108973 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21720d8-f812-4470-aba7-c63bf57d91a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.142038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.146817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.148094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.148586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.149081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.150033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:50.150560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.152148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.161340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 439/860] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:50.167394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.277241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432aa0-a508-410c-aa3c-06c4e1b85198 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.317246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.318209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.318495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.318770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.331107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.332635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-39393a35-be91-45ee-9012-d0818a55a1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:50.352890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:50.353843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:50.355675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:50.355675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:50.355675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:50.355675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:50.355675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/861] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:50.361976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.474688 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741f21e-8bb1-41a5-831a-cc7225d0b8d7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.504370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243bb6-b4bd-4b9f-ae59-167107fe1996 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.504713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243bb6-b4bd-4b9f-ae59-167107fe1996 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.504929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243bb6-b4bd-4b9f-ae59-167107fe1996 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.506747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/862] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:50.511096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.511637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.511759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.511941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.512121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.512217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.512427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.513092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:50.513405 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.514981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.529772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:50.530278 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 440/863] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:50.530885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:50.537253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.577342 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.601816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.625051 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/864] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:50.648072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.667425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc4b7e-c1a3-408c-bb5f-22aaf0cd597c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.709511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.710327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.710671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.710949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.720581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.723776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:50.743039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:50.744400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:50.745361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:50.747993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 441/865] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:50.752372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.777366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e034754-4f9c-461a-a7d8-a44a70b45036 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.820011 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.820641 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.820918 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.821279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.822840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.824648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.851949 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/866] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:39:50 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:39:50.893848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d84e46-2044-4152-899e-c7ff9dc3c5f2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.905740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.946518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.947202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.947481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.947751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.948627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:50.948935 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.950459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.979784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 442/867] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:50.982010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.023171 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:51.114304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783599f-0348-4007-a438-dc887343eba5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:51.153234 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:51.153895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.154184 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.154462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.234921 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:51.236927 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e52b1-afba-4f96-9072-4c151f12e303 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:51.269128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:51.369061 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/868] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:39:51.374454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.483596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd6062-00d2-4e33-9f28-bf254c43e419 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:51.520394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:51.521755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.522012 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.522376 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.534058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:51.535760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:51.535887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aa3-bb5a-4f8c-a493-1c5a96000566 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.536132 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aa3-bb5a-4f8c-a493-1c5a96000566 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.536471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a47aa3-bb5a-4f8c-a493-1c5a96000566 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.538500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/869] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:51.545938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.546236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.550705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:51.555178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:51.556137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:51.557034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:51.557034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:51.557034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:51.557034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:51.561065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 443/870] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:51.562660 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.570875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:51.572198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:51.579530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:51.672373 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/871] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:51.677536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11620aa-b1a3-4984-938e-300de4a35cd9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:51.689853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.714770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:51.715441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.715725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.716023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.716758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:51.717064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:51.718442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:51.736506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 444/872] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:51.737792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.765710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:51.824560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:51.827281 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12848aa4-7cba-4cdb-abab-567eefecc966 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:51.851004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d827-0463-47fb-8a19-bc7ae9efd287 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:51.866166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:51.866977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.867264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.867541 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.868237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:51.869622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f5769655-d8f3-4623-abbe-12ef2e6538c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:51.882436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:51.883493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:51.884415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:51.888335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/873] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:51.889392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8f92-bc52-4552-ac0b-8c63b2eb6bb5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.889678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8f92-bc52-4552-ac0b-8c63b2eb6bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.889953 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b8f92-bc52-4552-ac0b-8c63b2eb6bb5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.890787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:51.891380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/874] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:51.892323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.892613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b8f92-bc52-4552-ac0b-8c63b2eb6bb5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:51.893174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.893656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.903616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:51.905468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8b3a50d3-bcac-4275-9282-aa84959e209d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:51.922692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:51.923819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:51.924755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 445/875] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:51.925259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:51.925331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:51.925599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:51.944281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:51.945805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:52.008037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b8f92-bc52-4552-ac0b-8c63b2eb6bb5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.019216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 446/876] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:52.019862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:52.044431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:52.057891 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.058559 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.058842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.059119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.059823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:52.060162 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:52.061564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:52.071957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/877] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:39:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:52.071957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:52.092373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:52.136735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47220940-6fac-47fe-be76-cb3beeb8ebb9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.175183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.175780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.176227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.176516 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.177371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:52.177686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:52.178817 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0b2e-358a-4ac2-ba5e-afaaa4215817 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.179124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:52.192541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 447/878] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:52.192541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:52.221588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.222266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.222555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.222836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.223573 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:52.225058 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:52.248374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/879] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:39:52 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:52.249163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:52.323405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:52.331650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a6efa-82f7-4a73-87ed-34836680e64e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.369028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:52.394759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.396704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.396834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2e54-cf59-4126-b095-e9a0321eaa38 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.397344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.397825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.399115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:52.400349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:52.401763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45667dfa-33a5-4192-bbfe-6295ec4b7fd7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:52.408176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 448/880] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:39:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:52.449010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.449612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.449898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.450179 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.451048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=316eec64cd5f4184a0b97c946df81a54) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:52.451366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:52.452958 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8977e-87a2-4506-9e15-63d56bd1bb38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:52.461262 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/881] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:52 2022] GET /identity/v3/role_assignments?user.id=316eec64cd5f4184a0b97c946df81a54&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:53.243176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed6e9-dd5c-4a3f-b553-a6b4a7828d94 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.243558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed6e9-dd5c-4a3f-b553-a6b4a7828d94 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.243748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed6e9-dd5c-4a3f-b553-a6b4a7828d94 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.245403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 449/882] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:53.250554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.251011 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.251094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.268730 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:53.269922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:53.340290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/883] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:53.373735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed6e9-dd5c-4a3f-b553-a6b4a7828d94 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.480634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed6e9-dd5c-4a3f-b553-a6b4a7828d94 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.516368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.517091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.517378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.517656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.518316 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.519681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:53.532437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:53.533425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:53.534263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 450/884] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:53.541676 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.589880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:53.655228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d5bc9-41f9-4880-86e1-96876b189f0a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.694266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.695310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.695802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.696301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.697233 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:53.697742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.699319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.704904 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/885] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:53.710794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.763493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:53.817342 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579040fd-4e6f-4aa9-88c0-3352341ea789 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.853908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.854576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.854863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.855147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.855861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:53.856205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.857566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.862906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 451/886] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:53.869318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.973477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d950f-da6c-47f0-bd8b-053fd43f8aa4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.982572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:54.013016 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:54.014063 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.014561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.015036 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.016084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] RBAC: Adding query filter params (interface=public, service_id=14f1ebb284374102a9aee1e11eb24532, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:54.016601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:54.018192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54e083-173a-477b-badd-750e33a7fde0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:54.024616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/887] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/endpoints?service_id=14f1ebb284374102a9aee1e11eb24532&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:54.030289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:54.141038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33760f4-55cb-4412-90b7-eb0a0dbc89e0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:54.155494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:54.185519 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:54.186197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.186468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.186741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.187428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:54.188818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745cec9-996e-42a9-8238-1a0aa45dcf84 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:54.195863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 452/888] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:39:54 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:54.551775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d032e-4a45-4c64-b5f7-2a20f40e6d30 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.552593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d032e-4a45-4c64-b5f7-2a20f40e6d30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.552593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d032e-4a45-4c64-b5f7-2a20f40e6d30 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.555652 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/889] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:54.561576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8c1c-86c9-4060-8ac7-582a8f450341 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.562094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8c1c-86c9-4060-8ac7-582a8f450341 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.562196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8c1c-86c9-4060-8ac7-582a8f450341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.581165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7c4b8c1c-86c9-4060-8ac7-582a8f450341 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:54.582371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c4b8c1c-86c9-4060-8ac7-582a8f450341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:54.662559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 453/890] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:54.685689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d032e-4a45-4c64-b5f7-2a20f40e6d30 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:54.770391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef28ad-c583-4c20-b04f-5e3e796bebde None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.771101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef28ad-c583-4c20-b04f-5e3e796bebde None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.771409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef28ad-c583-4c20-b04f-5e3e796bebde None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.773080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 454/891] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:54.778130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f3c-9daa-47bb-9f39-4ed2b3c0c879 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.778397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f3c-9daa-47bb-9f39-4ed2b3c0c879 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.778683 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8f3c-9daa-47bb-9f39-4ed2b3c0c879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.794964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d032e-4a45-4c64-b5f7-2a20f40e6d30 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:54.796777 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-339e8f3c-9daa-47bb-9f39-4ed2b3c0c879 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:54.797964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-339e8f3c-9daa-47bb-9f39-4ed2b3c0c879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:54.833043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:54.834307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.834592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.834864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.835577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:54.837059 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-89557413-70c2-4213-ac3f-37a1aba7ffb5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:54.851673 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:54.852700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:54.854298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/892] 10.222.0.45 () {56 vars in 1132 bytes} [Wed May 25 13:39:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:54.855168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754bc4-b45b-476e-804c-c1cc81e1d5ee None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.855168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754bc4-b45b-476e-804c-c1cc81e1d5ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.855168 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754bc4-b45b-476e-804c-c1cc81e1d5ee None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.862886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/893] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:54.862886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754bc4-b45b-476e-804c-c1cc81e1d5ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:54.877352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 455/894] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:54.885988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:54.886762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:54.887653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:54.895010 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:54.909165 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:54.910326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:54.942412 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:54.972119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754bc4-b45b-476e-804c-c1cc81e1d5ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:54.991969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 456/895] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:54.991969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.028721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.029737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.030384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.030724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.031449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:55.031886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.033293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.043122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/896] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:55.043122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.097722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da39ff2-1817-4dfa-9054-f1e0a492b3d6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.135633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.136431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.136741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.136995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.137720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.139222 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.147566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.152435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c95e4-71d0-4061-8a94-8cfeb7f8d9e6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.154903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 457/897] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:39:54 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:55.155708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.193963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.209556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.210799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.211283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.211773 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.225380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:55.234607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:55.235628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:55.236903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:55.245618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/898] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:55.246924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.280987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af77e-7777-4099-af9d-e059cc13d0bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.322783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.323628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.323949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.324228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.324966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:55.325887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.326829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.338230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 458/899] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:55.339446 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.385624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12948649-acd2-401d-be39-f42f27ff8f78 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.433085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.433768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.434056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.434338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.435101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:55.435412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.436830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.446359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/900] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:55.452023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.461213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.474396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053a4d-f83c-4f0e-86f9-6208b7362da5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.496319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.512892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030bca-0347-4102-8538-10946ed347df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.513665 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030bca-0347-4102-8538-10946ed347df None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.514012 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030bca-0347-4102-8538-10946ed347df None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.514353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030bca-0347-4102-8538-10946ed347df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.515209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f030bca-0347-4102-8538-10946ed347df None admin] RBAC: Adding query filter params (interface=public, service_id=205b92f21ea1404ebd614b265f7f815d, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:55.515587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f030bca-0347-4102-8538-10946ed347df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.517059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f030bca-0347-4102-8538-10946ed347df None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.523276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 459/901] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/endpoints?service_id=205b92f21ea1404ebd614b265f7f815d&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:55.528608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030bca-0347-4102-8538-10946ed347df None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.589223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b56602-e234-462f-b4b6-0a788cadb563 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.635330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.636405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.636699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.636976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.651566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.653750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.662721 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030bca-0347-4102-8538-10946ed347df None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:55.673230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:55.674905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:55.676033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:55.676033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:55.676033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:55.676033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:55.676033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/902] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:55.682683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.709310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.710864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.711469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.711979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.713515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.715582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.724108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 460/903] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:55.764114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.803509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773db5-f45b-4501-8798-e463b01f94ca None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:55.844079 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:55.844762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.845050 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.845330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.846074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:55.846386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:55.847801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:55.861683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/904] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:55.867690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:55.890087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:55.985836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44454b0-5deb-49b1-ac3b-1a2000479fef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.022211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.023027 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.023312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.023589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.031627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.033097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:56.052893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:56.053906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:56.055273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 461/905] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:39:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:56.061119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.170935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a5425-33b7-4d42-8831-39ba5b30ce99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.198193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.211240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.211936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.212245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.212529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.213382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:56.213698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.215142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee3912-f514-4fde-823c-9396c0b0f6d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.234471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/906] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:56.239837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.329286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.356149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86d12-a597-4f8d-acc4-4ac2996943be None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.388905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.389558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.389846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.390128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.463380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.464835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.503450 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89b6-48d6-4596-8cb9-ea4738150276 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.503789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89b6-48d6-4596-8cb9-ea4738150276 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.504041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89b6-48d6-4596-8cb9-ea4738150276 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.505798 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/907] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:56.511081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.511641 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.512167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.530258 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:56.531756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:56.561145 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.591730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 462/908] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:56.597200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.612028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/909] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:56.631395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.726441 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985e9b5-f2d2-4d9e-8356-cf838ca6b8fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.740706 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc78da-2115-4833-a41d-ebcd1d11d99d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.758271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.771172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.772252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.772538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.772814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.779459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.780545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.781030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.781517 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.782383 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.783998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.784669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.786910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:56.797307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:56.798270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:39:56.799085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:56.806363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:56.807226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 463/910] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:56.808593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.809336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/911] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:56.812365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.846397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.923542 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb870-22fc-4563-bd32-4c7a91c54870 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.940013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6d70-fab5-46d3-8157-d8c6eec8df10 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.968697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.969523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.969822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.970221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.970964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:56.971277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.973421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.984836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.986228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.986870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.987414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.988315 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:56.988772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.989605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/912] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:56.990513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.003950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.004136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 464/913] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:57.007639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.054069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.121873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.133768 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d43a-6093-42ab-9cab-43aef483bda3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.139226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746fcac-0aaf-4400-ac36-6ee8e6374915 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.175319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.176128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.176409 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.176680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.184509 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.185489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.185962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.186625 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.187114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.187841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.189162 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:57.189667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.191215 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:57.202041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:57.202973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/914] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:57.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.204436 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 465/915] 10.222.0.45 () {56 vars in 1114 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:57.208005 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.318340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.321873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0a1a-4184-4dfd-93af-335147a4bb6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.336329 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0c2fe-f54c-484d-969d-2f2595d3f37f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.358929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.359609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.359989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.360336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.361193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.363024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.396402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/916] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:39:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:57.399570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.400296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.400579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.400895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.401809 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:57.402121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.403655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931a27c-2676-40eb-bd7e-7dfe00e00d88 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.418822 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 466/917] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:57.425031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.549686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21769491-ba70-48d3-8a82-d62811132ac7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.560033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3f59a-c5ce-46ed-9dca-6e1320d22a0f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.560371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3f59a-c5ce-46ed-9dca-6e1320d22a0f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.560593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3f59a-c5ce-46ed-9dca-6e1320d22a0f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.562593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 467/918] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:57.570929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.571285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.571744 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.589785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.590447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.590739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.590863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:57.591024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.591867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] RBAC: Adding query filter params (role.id=85ac4472b2064a8d8a1b23a9a2b609f4, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=316eec64cd5f4184a0b97c946df81a54) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:57.592233 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.592383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:57.593728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.601046 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/919] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/role_assignments?user.id=316eec64cd5f4184a0b97c946df81a54&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=85ac4472b2064a8d8a1b23a9a2b609f4 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:57.609456 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.680237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 468/920] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:57.703430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.729341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.822908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f24e8-cee1-48c4-9f2b-e95cf0170cc0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.860182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.860997 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.861282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.861561 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.862220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.863618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:57.876822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:57.879123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:39:57.881448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/921] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:57.889031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.999020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec01d2-b39e-4860-8524-0f3114585ede None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.036842 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.037864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.038400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.038880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.039792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:58.040353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.041956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eb482-28fd-4eb9-a7ab-5498333e0b8a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.047716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 469/922] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:58.054069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.132528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86916-8c42-47b7-8d23-910eebecf58e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.134006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86916-8c42-47b7-8d23-910eebecf58e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.134504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86916-8c42-47b7-8d23-910eebecf58e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.136708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 470/923] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:58.142031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.142581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.143063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.149095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.165217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:58.166527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008877fe-fafc-4ee0-8b17-2e1651a65edb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.167051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:58.170566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.205784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.206452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.206736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.207013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.207845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:58.208185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.209788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.215627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/924] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:58.221498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.256009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 471/925] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:58.275540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.341821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0af0-0791-42e5-a133-52e498123f42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.381205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72d1db-892c-4af7-836c-95e7e8ffd778 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.389600 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.390535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.391047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.391527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.392576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] RBAC: Adding query filter params (interface=public, service_id=7064a5db659a4b3bbe7072397e555fd2, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:58.393116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.394696 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.400999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/926] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/endpoints?service_id=7064a5db659a4b3bbe7072397e555fd2&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:58.406517 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.418002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.419080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.419570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.420060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.420923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.422478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.426500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:58.435969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:58.436941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:58.437794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:39:58.440712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 472/927] 10.222.0.45 () {56 vars in 1132 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:58.449034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.465096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.526943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822592-ebc6-4678-b798-8c3650db4f28 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.564895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.565925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.566428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.567020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.568052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.569757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.576837 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/928] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:58.588294 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faab3f2-088c-4893-9fde-34c07ca2f6dc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.629515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.630597 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.631096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.631525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.632458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:58.632959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.634608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.641257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 473/929] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:58.649196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.745866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.772902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec1dba-0289-4184-b04e-a69bbcd898de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.810144 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.810824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.811113 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.811395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.812228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:58.812547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.813940 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d40b16-e706-4470-a365-0b265af9f23f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:58.819772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/930] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:39:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:58.825627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:58.900548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:58.939747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05417f-5e55-4537-b373-d28c1e1b5bb0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:58.977181 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:58.978205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:58.978696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:58.979198 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:58.980207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:58.981766 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0dd01-3f6c-4612-8488-4cdf83b2a0f2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:59.004574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 474/931] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:39:58 2022] POST /identity/v3/endpoints => generated 347 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:59.847532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398825-f338-43ac-83f4-20e47a0a0b8b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:59.848555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398825-f338-43ac-83f4-20e47a0a0b8b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:59.849089 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83398825-f338-43ac-83f4-20e47a0a0b8b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:59.851985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/932] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:59.857670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:59.858211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:59.858710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:59.876371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913547-a7bb-4153-9468-a5d9c69c1d0a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:59.877009 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913547-a7bb-4153-9468-a5d9c69c1d0a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:59.877493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3913547-a7bb-4153-9468-a5d9c69c1d0a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:59.877920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:59.879102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:59.879870 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/933] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:59.888372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:59.888969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:59.889456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:59.908819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:59.909998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:59.976458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 475/934] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:59.992491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/935] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:39:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:59.993592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.018302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.060867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.096075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.133383 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c45953-f49e-4465-88fc-d149d636c8a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.135474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d3c2-7492-4818-be04-4e46feefcc56 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.172744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.173689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.173981 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.174378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.183617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.184812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.185295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.185739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.186901 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.187136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:00.188707 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.190274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING keystone.server.flask.common [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:00.196808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:00.197718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:00.198543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:00.203138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/936] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:00.204265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.216646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:40:00.216646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 13:40:00.216646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [parameters: {'id': '760fbb9f79e041c0836a195638d365e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 13:40:00.216646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 760fbb9f79e041c0836a195638d365e6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 760fbb9f79e041c0836a195638d365e6. May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:40:00.226531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 13:40:00.227345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() May 25 13:40:00.228293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:40:00.229166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:40:00.230039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:40:00.230827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': '760fbb9f79e041c0836a195638d365e6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.231667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:00.232475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 760fbb9f79e041c0836a195638d365e6. May 25 13:40:00.233390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.242976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 476/937] 10.222.0.45 () {60 vars in 1140 bytes} [Wed May 25 13:39:59 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:00.244741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.339663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012136-56c1-4344-9da4-49986dff0066 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.358865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3ea21-1d5f-4331-8c91-0efdf4173979 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.361650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.377098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.385414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.386024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.386313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.386595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.387526 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:00.387844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.389583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.406093 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/938] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:00.407809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.412284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.412775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.413061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.413346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.422782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.425097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:00.444876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:00.445810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:00.446685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:00.451252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 477/939] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:00.452385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.530717 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838e32-a26c-4d74-aace-318bbaf405d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.579257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.580236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.580674 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.580971 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.592163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13117fcd-94c7-4d49-8e22-0829513a59e1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.596879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.598426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:00.622970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:00.623907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:00.624822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:00.624822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:00.624822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:00.624822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:00.628959 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/940] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:00.629813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.635791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.636750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.637227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.637696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.638724 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:00.639225 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.641011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.676310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 478/941] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:00.686893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.753623 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f5d7-9115-4099-899d-5432dc2df9ba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.794443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.795511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.796027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.796500 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.797429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:00.797935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.799564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fe401-bc74-4243-8bc9-870a8c77acbd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.815022 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/942] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:00.820666 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.852258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.862518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75c75-3724-44cf-a0c4-36fb4c17cad5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.863737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75c75-3724-44cf-a0c4-36fb4c17cad5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.864285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75c75-3724-44cf-a0c4-36fb4c17cad5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.867891 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/943] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:00.876514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.877380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.877661 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.903762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:00.905431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:00.944559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434bb0b-b5d9-4dd4-9573-e21630844ab9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.991060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/944] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:00.992923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.993742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.994029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.994306 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.004621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.006622 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.017017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:01.023949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:01.024876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:01.025814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 479/945] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:01.030947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.056198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.141733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ca295-51e4-4fd4-963f-351f502e02b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.143903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.164911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05b57-bffe-444d-b5fb-f4f724c99eca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.186594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.187514 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.187854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.188655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.189396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.191129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.204207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.205183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.205651 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.206108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:01.206566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:01.207551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:01.209507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.211075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:01.211559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.213483 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.213747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/946] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:01.214595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.238715 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 480/947] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:01.240138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.336714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.339714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac45a10-8ee0-4875-afd3-d56bd6dcc943 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.374212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb292d-96cd-4fdb-be44-ae958fa9d708 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.377135 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.377821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.378126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.378412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.379156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:01.379487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.381506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905743ef-bd37-449f-ac01-c650a4f32f23 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.389745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/948] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:01.389745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed8e6-8e19-41d3-9592-e961b391a1e2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.389745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed8e6-8e19-41d3-9592-e961b391a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.389745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eed8e6-8e19-41d3-9592-e961b391a1e2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.389745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/949] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:01.391982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eed8e6-8e19-41d3-9592-e961b391a1e2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.418839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.419556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.419839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.420142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.431817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.433283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.435079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e899adc9-c0a7-4772-b419-7d45131df262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:01.454056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:01.454979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:01.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:01.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:01.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:01.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:01.455857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 481/950] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:01.456748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.456973 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.457254 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.475736 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:01.477570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:01.504482 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eed8e6-8e19-41d3-9592-e961b391a1e2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.540150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.540816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.541101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.541376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.542082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:01.542395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.543765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.553838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/951] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:01.553838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.554111 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 482/952] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:01.555684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.589187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.669090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700cd3f-8820-4063-9524-155bd61f907a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.674568 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abcb44-cc84-46f0-9c7d-8c49fc15924f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.694582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.707667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.708876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.709172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.709538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.710293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.710594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.711255 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.711544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.711650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.711824 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.712584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:01.712895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.714256 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.727827 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/953] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:01.728579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.736861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 483/954] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:40:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:01.737678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.836273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:01.854786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b6b0f-a74c-440f-bd66-9883bee3ae4e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.865153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dfcaa-67df-4c3e-98bc-f741dedddcdd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.894586 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.895404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.895714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.896041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.896828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.898230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.907308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.908508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.909002 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.909515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:01.912284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:01.913579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:01.914557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/955] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:01.922363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:01.928390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.931526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:01.949675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:01.950641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:01.951547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 484/956] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:01.958034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:02.002209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:02.058031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad13984-fed9-4b01-8eeb-719b0b2fc3c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:02.080211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd6da6-530f-4cd0-8e59-ba4658dc0550 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:02.105847 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:02.106778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:02.107072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:02.107326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:02.108231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:02.108852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:02.110682 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:02.117599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/957] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:02.119830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:02.120527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:02.120813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:02.121262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:02.122336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:02.122662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:02.123442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:02.124122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:02.141671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 485/958] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:02.147062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:02.159318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:02.249175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded41ed-025d-4bf4-b674-c18660a2aa3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:02.276554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84bf8-5ddc-43f6-81ed-3b7cc8ffa712 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:02.280238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:02.285193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:02.286136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:02.286695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:02.287187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:02.288172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:02.288744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:02.290504 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9d851-c1e2-4626-841d-ab94a61a3999 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:02.295765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/959] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:02.325378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:02.326290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:02.326572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:02.326851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/316eec64cd5f4184a0b97c946df81a54/roles/85ac4472b2064a8d8a1b23a9a2b609f4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:02.329275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=316eec64cd5f4184a0b97c946df81a54, role_id=85ac4472b2064a8d8a1b23a9a2b609f4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:02.330748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d09a9-96a1-458e-97ce-2301aab786de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:02.355971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 486/960] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:40:02 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/316eec64cd5f4184a0b97c946df81a54/roles/85ac4472b2064a8d8a1b23a9a2b609f4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:03.107898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db299-be44-434b-9b70-6b76994b123f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.108431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db299-be44-434b-9b70-6b76994b123f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.108659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db299-be44-434b-9b70-6b76994b123f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.110140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/961] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:03.116425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.116874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.117054 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.140494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:03.141759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:03.217390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 487/962] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:03.242642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db299-be44-434b-9b70-6b76994b123f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:03.328166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:03.359013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db299-be44-434b-9b70-6b76994b123f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:03.393229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:03.393904 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.394186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.394463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.395176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:03.397158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:03.398465 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 13:40:03.412506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:40:03.412506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 13:40:03.412506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [parameters: {'user_id': '4a5bf1f77c5c49b2be709d22d2b0832a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 13:40:03.412506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() May 25 13:40:03.429325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:40:03.430278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:40:03.431239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:40:03.432187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:40:03.433100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:40:03.433962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'user_id': '4a5bf1f77c5c49b2be709d22d2b0832a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:03.434804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 13:40:03.435591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:03.440688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/963] 10.222.0.45 () {60 vars in 1131 bytes} [Wed May 25 13:40:03 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:03.441104 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:03.532286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:03.552861 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f73f6-5e7d-4b1f-bef1-2c66055abfd6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:03.589547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:03.590723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.591212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.591716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.604605 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:03.605552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4b709024-0f71-4200-8a78-0b1b29ad2714 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:03.621773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:03.622758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:03.623805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 488/964] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:03.633402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:03.738371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b5ef-c75b-40f2-af2e-9604386d9b0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:03.774071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:03.774747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.775043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.775322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.776157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:03.776470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:03.778163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fad812-1686-4ab9-aaa9-c18a7d49b953 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:03.802550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:03.817281 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/965] 10.222.0.45 () {56 vars in 1101 bytes} [Wed May 25 13:40:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:04.135903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49a51-3e27-4b5e-aff7-29057e9bd138 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.136465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49a51-3e27-4b5e-aff7-29057e9bd138 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.136546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c49a51-3e27-4b5e-aff7-29057e9bd138 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.137915 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 489/966] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:04.143310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.143706 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.143875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.162566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:04.163880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:04.234697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/967] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:04.258092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49a51-3e27-4b5e-aff7-29057e9bd138 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.369475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c49a51-3e27-4b5e-aff7-29057e9bd138 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:04.404262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:04.404991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.405273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.405547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.406202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:04.407557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:04.420266 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:04.421282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:04.422155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 490/968] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:40:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:04.429315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.543339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcd501-8564-459a-a924-1c502669d6c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:04.571692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:04.580584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:04.581257 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.581539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.581814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.582521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:04.582829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:04.585348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b801-c567-44ad-8145-7125596e6fcb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:04.594873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/969] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:04.596477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.610531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:04.632807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ec5c-5571-49bf-a13c-cda24bc2d590 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.633117 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ec5c-5571-49bf-a13c-cda24bc2d590 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.633380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ec5c-5571-49bf-a13c-cda24bc2d590 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.634913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/970] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:04.640791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d4c0-1fe4-4f81-8903-55858a9f5cc8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.641174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d4c0-1fe4-4f81-8903-55858a9f5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.641457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d4c0-1fe4-4f81-8903-55858a9f5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.661064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0169d4c0-1fe4-4f81-8903-55858a9f5cc8 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:04.662728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0169d4c0-1fe4-4f81-8903-55858a9f5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:04.710056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ceadc-2288-4b1b-aee8-2e51ad513b04 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:04.731231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/971] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:04.732576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7de629-7584-4559-ae0a-33f1e9e10877 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.732868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7de629-7584-4559-ae0a-33f1e9e10877 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.733151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7de629-7584-4559-ae0a-33f1e9e10877 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.734542 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/972] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:04.739582 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.739868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.740247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.748687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:04.749354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.749637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.749911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.751044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:04.751362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:04.752979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:04.756647 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:04.757828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:04.758245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 491/973] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:04.758944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.824954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/974] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:04.826064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.844101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:04.870885 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5dd9-54ff-4a10-869c-87a2f7d4912d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:04.882394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:04.904500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:04.905153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.905443 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.905719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.906438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:04.907781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:04.920658 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 492/975] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:40:04 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:04.921539 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:04.940167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac7cf8-a6b8-4d01-9399-52d95cf85e0f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:04.988100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:04.988785 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:04.989069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:04.989350 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:04.990807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] RBAC: Adding query filter params (interface=admin, service_id=7064a5db659a4b3bbe7072397e555fd2, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:04.991235 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:04.992709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.002992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/976] 10.222.0.45 () {56 vars in 1249 bytes} [Wed May 25 13:40:04 2022] GET /identity/v3/endpoints?service_id=7064a5db659a4b3bbe7072397e555fd2&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:05.004288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.042416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710806-9149-42e0-ba6d-6224c506f40d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.078981 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.079812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.080122 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.080394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.082589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:05.096586 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:05.105709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:05.106700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:05.107554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 493/977] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:40:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:05.112716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.119245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d0c71-7370-4eda-b8b5-f5c83d08bc23 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.134709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:05.158922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.159701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.159885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.160206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.160897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.162503 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.169031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/978] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 13:40:05.235455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e6df-6d9b-4233-989a-747d0b077962 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.273569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.274255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.274542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.274821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.275585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:05.275896 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.277326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.291664 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 494/979] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:05.293485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.411972 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047c4f4-8857-4384-9c18-35a9fa42045f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.446760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.447559 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.447843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.448146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.459892 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.461419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:05.479196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:05.480371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:05.485896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:05.485896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:05.485896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:05.485896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:05.485896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/980] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:05.487619 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.545999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:05.591840 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19cd4a-d4f4-46b7-82ab-b1fccbbc12ba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.625830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.626496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.626780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.627058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.627787 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:05.628124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.629496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.646728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 495/981] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:05.648549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.678951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:05.754335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e1840-641f-41df-8ff3-1644e169e368 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.789435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.790511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.790944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.791385 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.799201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.800839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:05.815510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:05.816545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:05.817477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/982] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:05.824821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:05.926957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d916c-eb56-4969-b07a-77a8cda6cb78 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:05.944430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:05.963634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:05.964546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:05.965081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:05.965511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:05.966670 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:05.967144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:05.968759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:05.992642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 496/983] 10.222.0.45 () {56 vars in 1149 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:05.994183 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.086584 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:06.101761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f6113-17d7-4b89-b868-46d09e3581a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.136914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.137722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.138111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.138490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.207612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.209786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.295391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:06.314681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/984] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:05 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:40:06.315907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.420161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3241276-f318-4f3b-bc0f-fb065297a184 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.453908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.454694 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.454977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.455251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.465531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.466967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:06.485154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:06.486048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:06.491498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:06.491498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:06.491498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:06.491498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:06.491498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 497/985] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:06.492193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.591591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c6db2-3b52-4ff4-860c-41b77e270871 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.626205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.626866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.627148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.627423 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.628164 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:06.628474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.629840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.645505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/986] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:06.646920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.676118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:06.754337 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76357ec-c81a-4b88-8ca2-22227810bdec None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.788944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.789732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.790018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.790297 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.797116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.798544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:06.813299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:06.814217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:06.815087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:06.819793 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 498/987] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:06.821191 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.873588 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:06.925941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99900b-e4f4-415d-ab08-1d717718b8bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.960851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.961518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.961801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.962073 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.962914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:06.963226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.964694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41ec53-e1e0-46a3-b5a9-5d43a70b6a33 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.982436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/988] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:06.984029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.077476 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.094969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656ebc-c5e2-470f-b9e1-dfff00e847b5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.129354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.130237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.131242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.131581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.132525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] RBAC: Adding query filter params (role.id=85ac4472b2064a8d8a1b23a9a2b609f4, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=316eec64cd5f4184a0b97c946df81a54) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:07.132908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.134393 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a82be-fd46-4521-b223-344100128934 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.148209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 499/989] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/role_assignments?user.id=316eec64cd5f4184a0b97c946df81a54&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=85ac4472b2064a8d8a1b23a9a2b609f4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:07.188060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16f6-09d1-4962-808a-e07c8cc9ee66 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.188275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16f6-09d1-4962-808a-e07c8cc9ee66 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.188556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c16f6-09d1-4962-808a-e07c8cc9ee66 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.190155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/990] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:07.196384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7334-ab65-4e29-b164-1a3de99f96e6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.196686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7334-ab65-4e29-b164-1a3de99f96e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.196957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7334-ab65-4e29-b164-1a3de99f96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.215178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9a2e7334-ab65-4e29-b164-1a3de99f96e6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:07.216468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9a2e7334-ab65-4e29-b164-1a3de99f96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:07.295876 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 500/991] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:07.319303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c16f6-09d1-4962-808a-e07c8cc9ee66 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.430852 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf61a-1884-455e-9273-e15aae7910a6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.431386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf61a-1884-455e-9273-e15aae7910a6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.431386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf61a-1884-455e-9273-e15aae7910a6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.432873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 501/992] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:07.438253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.438485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.438769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.439056 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c16f6-09d1-4962-808a-e07c8cc9ee66 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.450216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.457199 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:07.458426 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:07.479353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.480187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.480476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.480751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.481410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.482792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:07.498523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:07.499543 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:07.500445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/993] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:07.505799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.531888 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 502/994] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:07.554652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.566684 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.617292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf68f62-0b5e-4033-9b80-73fed902f6d4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.653433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.654101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.654389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.654667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.654789 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791faa-f4d8-494f-a152-980d2d66ce2d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.655377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:07.655684 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.657106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.662694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/995] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:07.668222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.699511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.700263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.700554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.700834 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.701533 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.702857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.705709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:07.716013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:07.717227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:07.718081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 503/996] 10.222.0.45 () {56 vars in 1126 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:07.722997 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.783354 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abe1c0-2d02-40ab-976d-baec41631164 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.800674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.825657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.826323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.826610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.826889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.827591 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:07.827897 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.828877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63deeb-518b-45ea-8714-c94cb57fd64b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.829296 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.836264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/997] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:07.840851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.866435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.867102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.867391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.867671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.868425 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:07.868734 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.870132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.875343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 504/998] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:07.880771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:07.950015 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:07.952874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2113fa-d458-49ee-b118-a93d82e79dec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.992277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb7348-8448-4148-8a3a-421db41b120c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:07.993186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:07.993919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:07.994289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:07.994625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:07.995485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] RBAC: Adding query filter params (interface=public, service_id=a640f98a9d8846c59befb0a829ddcdf7, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:07.995888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:07.997343 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:08.007862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/999] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/endpoints?service_id=a640f98a9d8846c59befb0a829ddcdf7&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:08.008809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:08.030986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:08.032173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:08.032541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:08.032847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:08.033534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:08.033844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:08.035201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:08.040704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 505/1000] 10.222.0.45 () {56 vars in 1122 bytes} [Wed May 25 13:40:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:08.046149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:08.049095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:08.120473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518376c-09a7-4811-8407-24076029ccb1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:08.147943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a627d-cc9c-4c03-bc6d-83db8bccb202 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:08.155781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:08.156467 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:08.157037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:08.157037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:08.157681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:08.159065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dc172-8f7f-44ad-bc20-1951ea087300 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:08.166194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/1001] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:40:08 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:40:08.184491 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:08.185271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:08.185621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:08.185961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:08.186800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:08.188305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4229-b163-46c5-a945-2bde1a9bc658 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:08.212063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 506/1002] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:40:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:09.425333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87550886-05b8-466a-a6e6-6f48fc0a6e7e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.425880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87550886-05b8-466a-a6e6-6f48fc0a6e7e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.425880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87550886-05b8-466a-a6e6-6f48fc0a6e7e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.431712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/1003] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:09.433654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.434395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.434682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.452489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:09.453663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:09.515013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:09.533626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 507/1004] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:09.556923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87550886-05b8-466a-a6e6-6f48fc0a6e7e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:09.588823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:09.667104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87550886-05b8-466a-a6e6-6f48fc0a6e7e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:09.702425 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:09.703207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.703489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.703760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.704455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:09.705808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:09.717712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:09.718784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:09.719614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:09.721344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/1005] 10.222.0.45 () {56 vars in 1105 bytes} [Wed May 25 13:40:09 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:09.725816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:09.836835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a627a77-843a-4383-88e2-8f5804e3a70a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:09.872135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:09.872799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.873079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.873377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.874058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:09.874361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:09.875718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ed15d-a77b-4639-bfcf-e8a82c655242 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:09.885693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 508/1006] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:09.887216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:09.983070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:09.993559 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bee72-0f84-41ad-95eb-fc6a5c2ef287 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:10.028483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:10.029144 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.029426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.029701 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.030405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:10.030712 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:10.032102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421d103-5c7f-4430-87e9-622feef1853a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:10.044033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/1007] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:10.408496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc51286-ad53-45f9-ad5d-749aa619d180 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.408797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc51286-ad53-45f9-ad5d-749aa619d180 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.409061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc51286-ad53-45f9-ad5d-749aa619d180 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.410592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 509/1008] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:10.416217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.416518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.416802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.433471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:10.434674 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:10.499899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069291f-b1fe-4715-8ab2-b16d61499515 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.500432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069291f-b1fe-4715-8ab2-b16d61499515 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.500556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069291f-b1fe-4715-8ab2-b16d61499515 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.501953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 510/1009] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:10.504570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/1010] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:10.507606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.507898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.508205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.524943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:10.526101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:10.527958 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:10.582515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:10.605880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 511/1011] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:10.626527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:10.629108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:10.642845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439ff17-048f-4f23-b0be-9a729ed37e20 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:10.678594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:10.679396 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.679686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.679984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.680647 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:10.682024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:10.694715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:10.695736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:10.696695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/1012] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:10.701723 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:10.739907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167e067-2d49-45c3-b0ff-08d1be62dd66 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:10.777461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:10.778264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.778541 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.778817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.780006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:10.781424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:10.783193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 512/1013] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:10.789067 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:10.816069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429ef7e-2bb3-4713-a3c0-c9a080d89a92 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:10.842630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:10.853080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:10.853757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.854070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.854349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.855078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:10.855374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:10.856846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:10.866965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/1014] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:10.868047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:10.885445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:10.897709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607959-f853-4460-8b2f-0ef3e53fc6a0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:10.936918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:10.937574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:10.937859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:10.938136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:10.938843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:10.940285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:10.968332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 513/1015] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:40:10 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:10.978298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035155f-986d-480b-815a-873bcdd7bfa1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:11.017778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:11.018455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:11.018742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:11.019019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:11.019736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:11.020083 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:11.021637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c419613-f5c2-453b-810e-e583845fc538 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:11.032016 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/1016] 10.222.0.45 () {56 vars in 1112 bytes} [Wed May 25 13:40:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:11.033926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:11.144844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51656464-274c-461d-bcd7-4f700822ce2d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:11.147360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:11.185903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:11.186595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:11.186887 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:11.187166 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:11.187964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:11.189332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a55bd-0283-4b9a-8428-6d4b7bf9668c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:11.211296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 514/1017] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:40:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:12.271515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ae90-2024-411c-a132-be2f37089328 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.273203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ae90-2024-411c-a132-be2f37089328 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.273203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ae90-2024-411c-a132-be2f37089328 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.273603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/1018] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:12.278863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3a70-7469-4697-b850-5b02028652e2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.279254 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3a70-7469-4697-b850-5b02028652e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.279455 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3a70-7469-4697-b850-5b02028652e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.296282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e8a3a70-7469-4697-b850-5b02028652e2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:12.297465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e8a3a70-7469-4697-b850-5b02028652e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:12.379488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 515/1019] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:12 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:12.402721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20ae90-2024-411c-a132-be2f37089328 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:12.486852 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:12.511329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20ae90-2024-411c-a132-be2f37089328 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:12.548763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:12.549349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.549630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.549903 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.550642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:12.552091 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44de20-6eb7-4f6e-89c8-516aa360e95d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:12.572707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/1020] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:40:12 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:13.264948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6bf5-436c-4948-84bf-37266becc1b3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.265332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6bf5-436c-4948-84bf-37266becc1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.265518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6bf5-436c-4948-84bf-37266becc1b3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.266929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 516/1021] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:13.272322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358907-42b8-4b53-92dc-f2916da67375 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.272756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358907-42b8-4b53-92dc-f2916da67375 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.272957 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358907-42b8-4b53-92dc-f2916da67375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.290030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53358907-42b8-4b53-92dc-f2916da67375 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:13.291220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53358907-42b8-4b53-92dc-f2916da67375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:13.374466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:13.399071 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6bf5-436c-4948-84bf-37266becc1b3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.488408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.508147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6bf5-436c-4948-84bf-37266becc1b3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.527160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b6a8-f4a1-4d81-9858-5231db7ab7d0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.527384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b6a8-f4a1-4d81-9858-5231db7ab7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.527661 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b6a8-f4a1-4d81-9858-5231db7ab7d0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.529197 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/1023] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:13.533782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.534074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.534359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.545299 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.546090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.546374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.546649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.547819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.549298 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.555528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 517/1024] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:13.555675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:13.555675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:13.556983 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.614721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.632261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/1025] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:13.648690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.672725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e17bf-2c99-410f-ac80-202bf48ed475 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.714506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.715178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.715464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.715970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.716489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.717907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.740042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 518/1026] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:40:13 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:13.759866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4e15d-fc79-402b-a865-7f876382da3a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.798366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.799122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.799398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.799677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.812572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:13.821199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:13.822261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:13.823115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/1027] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:13.828585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.841534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.940011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20043dce-63b0-4c74-aa12-8f729afb1f50 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.976663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.977332 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.977623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.977895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.978640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:13.978948 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.980328 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.988975 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 519/1028] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:13.994360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:14.055140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:14.103686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ab020-61fd-4d68-b017-f0c8d7962b78 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:14.138751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:14.139544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.139827 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.140143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.151154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:14.152737 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:14.170809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:14.171767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:14.173099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:14.173099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:14.173099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:14.173099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:14.173099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/1029] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:14.178108 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:14.248665 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:14.282051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476145b-9186-4253-bd0a-24a6a911dbbb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:14.316881 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:14.318318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.318538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.318635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.319340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:14.319649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:14.321128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:14.333737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 520/1030] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:14.339029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:14.445946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921437d7-c2ca-4ca2-a913-0c304e5ebead None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:14.453437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:14.483642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:14.484780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.485246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.485699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.493404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:14.495038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:14.509663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:14.510630 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:14.511575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:14.517929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1031] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:40:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:14.519183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:14.629292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04804f1f-d4e3-4fe2-911b-5f0b72333e83 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:14.660758 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:14.668520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:14.669198 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.669485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.669762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.670608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:14.670922 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:14.672384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2c23-17df-42c5-9c16-b5180e3e9adf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:14.695075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 521/1032] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:40:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:14.696381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:14.799101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c5487-ee98-467a-88f3-ab201d6a8d3e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:14.810292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b457d-fea1-4e5c-b8c8-0d327488b7b0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.810554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b457d-fea1-4e5c-b8c8-0d327488b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.810833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b457d-fea1-4e5c-b8c8-0d327488b7b0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.815749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 522/1033] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:14.817988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.818774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.819061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.836630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:14.837811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:14.842627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:14.843282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:14.843563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:14.843839 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:14.881950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:14.913650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 523/1034] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:14.922619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:14.924205 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:14.935831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.042638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1035] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:15.044777 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f2837-1e76-4ef6-b351-18a0aa22f538 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.048321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.064751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.084669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.085458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.085739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.086015 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.086671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.088079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:15.101489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:15.102537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:15.104281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 524/1036] 10.222.0.45 () {56 vars in 1105 bytes} [Wed May 25 13:40:14 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:15.108508 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.119589 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.159981 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91604ef-e38c-414a-b355-123ac35a3dff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.197976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.198871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.199228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.199620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.210263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.211741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.217161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b27015-d1b9-45f3-8fc4-6c870650d577 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:15.236445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:15.237464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:15.238626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:15.238626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:15.238626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:15.238626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:15.240774 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1037] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:15.246155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.254124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.255817 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.256336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.256621 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.257696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.258008 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.259682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.269892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 525/1038] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:15.272095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.333382 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.371530 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0ad3-3c64-40b2-a39a-42210d4c1ac5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.391969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.405318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ce725-99e4-469c-b0fa-93109c110094 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.409774 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.410487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.410724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.411001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.412549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.412820 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.414217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.426363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1039] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:15.432161 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.444064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.444816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.445099 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.445373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.446204 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.446391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.447743 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.457587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 526/1040] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:15.459751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.537546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72bdb9-d389-4235-b9c2-922bf214bc3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.565955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d342-9e53-49e4-ba60-7416f347740b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.573281 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860729-65f2-4441-994d-d0441555a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.574319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860729-65f2-4441-994d-d0441555a99a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.574601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860729-65f2-4441-994d-d0441555a99a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.574878 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860729-65f2-4441-994d-d0441555a99a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.578646 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.582003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18860729-65f2-4441-994d-d0441555a99a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.583489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18860729-65f2-4441-994d-d0441555a99a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-18860729-65f2-4441-994d-d0441555a99a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:15.598180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:15.599965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:15.600941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1041] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:15.605756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860729-65f2-4441-994d-d0441555a99a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.610725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.611614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.612170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.612631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.613657 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] RBAC: Adding query filter params (interface=public, service_id=10771a2af71549138e97e8ab5cd75707, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.614154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.616319 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.621979 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 527/1042] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/endpoints?service_id=10771a2af71549138e97e8ab5cd75707&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:15.627388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.652027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.716894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860729-65f2-4441-994d-d0441555a99a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.738440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2b408-3d82-4f39-bfaa-8a4aca6edd8c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.753555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.754468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.754944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.755272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.756154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.756466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.757893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.770118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1043] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:15.777105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:15.781381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.782272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.782730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.783161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.784026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.785530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daff61-1fd5-4bb1-9f84-6723401e46be None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.794349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 528/1044] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:40:15.887281 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:15.892642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caea549-8d06-4264-b630-74d194f3f1ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:15.931432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:15.932137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:15.932429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:15.932709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:15.933563 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] RBAC: Adding query filter params (role.id=85ac4472b2064a8d8a1b23a9a2b609f4, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:15.933885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:15.935364 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc325ee4-6588-4fc5-8d6d-192593e7a3bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:15.948066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1045] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:15 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=85ac4472b2064a8d8a1b23a9a2b609f4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:16.069752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8804b-e040-4b0d-b14f-77a19b3103bd None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.070064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8804b-e040-4b0d-b14f-77a19b3103bd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.070230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8804b-e040-4b0d-b14f-77a19b3103bd None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.071796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 529/1046] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:16.077535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.077905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.078264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.096902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:16.098147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:16.176540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1047] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:16.200133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8804b-e040-4b0d-b14f-77a19b3103bd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.242458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.307739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8804b-e040-4b0d-b14f-77a19b3103bd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.348059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.349865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.349865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.350192 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.366454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:16.374069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:16.375018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:16.375864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 530/1048] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:16.381289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.475729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.486115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf633f6f-cec1-491a-adb8-29a5526c15ba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.520421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.521159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.521471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.521717 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.522465 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:16.522778 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:16.524178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:16.537065 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1049] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:16.538284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.641844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.647537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e163-6bb8-440d-bc03-8893db3da2a6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.683173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.684075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.684362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.684640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] PATH_INFO: `/v3/users/b6064d3e677549119a0e49e7affbc0fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.698326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] RBAC: Authorizing `identity:get_user(user_id=b6064d3e677549119a0e49e7affbc0fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:16.699756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:16.705964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 531/1050] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/users/b6064d3e677549119a0e49e7affbc0fb => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:16.707130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.808107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f3ae-fae0-40d8-96fe-3bfaea50ffeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.841486 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.842128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.842412 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.842686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.866748 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.911116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:16.913101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.018544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1051] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:40:17.019863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:17.125907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4e99-4623-4aae-b747-89a8eb4c0b47 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:17.161092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:17.162100 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:17.162331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:17.162608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:17.163796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:17.165270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc47490-abde-46f5-89fe-9c0cf210434b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.167129 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 532/1052] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:17 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:17.172976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:17.230472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:17.276023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c4aad-c5b9-46ff-acbe-6ca0749a0981 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:17.310225 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:17.310876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:17.311160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:17.311435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:17.312295 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=e23857b9290e4dd9b8ce4f098c168bca, user.id=b6064d3e677549119a0e49e7affbc0fb) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:17.312613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:17.314049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428c43-8688-4d76-90ce-7f878f503c81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.328263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1053] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:17 2022] GET /identity/v3/role_assignments?user.id=b6064d3e677549119a0e49e7affbc0fb&scope.project.id=e23857b9290e4dd9b8ce4f098c168bca&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:18.096179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1625-9db7-48ed-99ed-e628bbc397e6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.096705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1625-9db7-48ed-99ed-e628bbc397e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.096705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1625-9db7-48ed-99ed-e628bbc397e6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.098253 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 533/1054] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:18.104076 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.104516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.104732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.121906 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:18.123111 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:18.141733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269f0dd-3b72-488b-8ea2-2955ea03e6ef None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.142028 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269f0dd-3b72-488b-8ea2-2955ea03e6ef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.142310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269f0dd-3b72-488b-8ea2-2955ea03e6ef None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.143807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 534/1055] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:18.148988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.149277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.149553 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.167847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:18.169918 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:18.191082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.197037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1056] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:18.219845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.245023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 535/1057] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:18.267246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.274063 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.337088 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc3c56-d977-441d-80b2-993a52d0d0ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.374967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd66954-b5a6-443d-a61f-a9ffc9b4c83e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.379707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.380645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.380898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.381181 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.381838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.383222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:18.394916 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:18.397129 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:18.397976 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1058] 10.222.0.45 () {56 vars in 1105 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:18.402633 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.423042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.424268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.424844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.425323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.437582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:18.448125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:18.449072 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:18.449904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 536/1059] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:18.455513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.473115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.511392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc5ac6-3003-4912-9fb6-c3d191592440 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.525819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.549326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.550115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.550401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.550813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.551507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:18.551859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.553286 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.560080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1060] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:18.565007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.571728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89c521-f69e-4c01-a20c-97ae9d580029 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.609863 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.611337 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.611834 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.612343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.613774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:18.614713 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.616709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.629356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 537/1061] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:18.634624 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.680620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e743c-2476-42f9-87ce-9f0ccb25f71d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.717735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.720458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.721454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.721759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.722039 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.722760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:18.723449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.725643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.733801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1062] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:18.739292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.745939 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6b2f-41dc-486a-ba88-c3f8e5768bf6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.782821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:18.783084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.783846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.784293 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.784467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.796145 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.797593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:18.818230 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:18.819209 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:18.820735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:18.820735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:18.820735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:18.820735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:18.821312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 538/1063] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:18.825040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:18.855802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d3ec-7873-461c-bfcd-c6f392090c37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.891553 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:18.892468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:18.892941 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:18.893370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:18.894251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:18.896349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:18.916490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1064] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:40:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:18.952729 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b1a1e-3267-4418-a0a3-17b4c671beec None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:18.986018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.003406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.004112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.004401 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.004667 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.006082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:19.006398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.007778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.022578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 539/1065] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:19.028115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.134095 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.141358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4511b4b-ca3c-48e0-9a99-f0e2dae8919f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.178637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.179338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.179616 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.179887 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.187565 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.189033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:19.203867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:19.204954 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:19.205893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1066] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:19.213418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.328486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd6d03-9b62-45d1-b42d-d3d86db9fa78 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.377825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.378668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.379055 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.379432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.380963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:19.381384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.382994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce5870-1a1c-4bab-bfe6-fdab6285f899 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.405148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 540/1067] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:19.406511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.513755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf644e2-1465-4f26-bab5-58e8d6d14e6f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.525034 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d9ab-ce6e-49fb-8828-ced0b4a55008 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.525334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d9ab-ce6e-49fb-8828-ced0b4a55008 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.525615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5d9ab-ce6e-49fb-8828-ced0b4a55008 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.527124 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 541/1068] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:19.531764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.532086 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.532363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.548609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:19.549653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.549783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:19.550638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.551023 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.551401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.564710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.566172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:19.584690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:19.585769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:19.586883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:19.586883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:19.586883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:19.586883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:19.586883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1069] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:19.591252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.591371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.603886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.627275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 542/1070] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:19.649731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.697229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99e9c0-e85e-4168-85e9-d80c7dc3e869 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.733516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.734178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.734463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.734736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.735460 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:19.735770 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.737179 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.751434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1071] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:19.757478 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.759490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bb32f-3095-403d-9c95-a926213c3d8c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.799545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.800522 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.800803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.801141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.812617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:19.821417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:19.822554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:19.823584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:19.824610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 543/1072] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:19.830269 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.832044 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.863320 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.867879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443d56-3604-424e-ba3b-baa7c63f14e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.902961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.903865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.904243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.904587 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.911457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.913067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:19.927196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:19.928302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:19.929284 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1073] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:19.934109 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:19.939370 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55afe3fe-af6d-4585-97d7-d173f5091fd3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:19.978200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:19.979186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.979706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.980334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.981390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:19.981917 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:19.983589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:19.993384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 544/1074] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:19.998628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.039848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c027c9-cd84-4273-b434-992ecda509ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.065180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.074366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.075193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.075575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.075973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.076889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:20.077304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.078889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.099143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1075] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:20.100189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.109551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.117786 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55c08c-ae5a-48fa-9bb3-4a92362d0732 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.156102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.156973 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.157254 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.157526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] PATH_INFO: `/v3/users/b6064d3e677549119a0e49e7affbc0fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.159095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] RBAC: Authorizing `identity:get_user(user_id=b6064d3e677549119a0e49e7affbc0fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.160562 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.167351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 545/1076] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/users/b6064d3e677549119a0e49e7affbc0fb => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:20.168070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.206610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96e33-2f1d-4f94-be12-44e1566e2c9e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.242909 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.243797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.244105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.244375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/85ac4472b2064a8d8a1b23a9a2b609f4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.246822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=6086ccae239045e5b109e33cf47453a2, role_id=85ac4472b2064a8d8a1b23a9a2b609f4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.248398 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.270670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1077] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:40:20 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6086ccae239045e5b109e33cf47453a2/roles/85ac4472b2064a8d8a1b23a9a2b609f4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:20.271263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a15d5-69b5-4687-a9ec-5a1f08b11569 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.310466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.311414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.311637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.311914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.313189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.314639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78066f-ce15-479b-ba09-a414f9d9f2dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.316748 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 546/1078] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:20.323237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.386580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.446814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8356d-0239-4a95-8730-a672ad87ffae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.483241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.484153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.484437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.484713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca/users/b6064d3e677549119a0e49e7affbc0fb/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.486658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] RBAC: Authorizing `identity:create_grant(project_id=e23857b9290e4dd9b8ce4f098c168bca, user_id=b6064d3e677549119a0e49e7affbc0fb, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.488123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2851fac-bd06-4f8a-a35a-f2c8d6fa426c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.501548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1079] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:40:20 2022] PUT /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca/users/b6064d3e677549119a0e49e7affbc0fb/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:21.199348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34713e-217a-4541-9aa4-d5274bc85943 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.199795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34713e-217a-4541-9aa4-d5274bc85943 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.200071 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34713e-217a-4541-9aa4-d5274bc85943 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.201633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 547/1080] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:21.206705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.207157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.207263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.225249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:21.226688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:21.293143 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1081] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:21.315738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34713e-217a-4541-9aa4-d5274bc85943 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:21.352188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:21.433463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34713e-217a-4541-9aa4-d5274bc85943 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:21.467442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:21.468048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.468339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.468618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.538106 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:21.539565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:21.568704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:21.643820 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 548/1082] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:21.650317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:21.691817 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:21.756513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149dbc4-1abc-4a02-b584-f7477715854e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:21.790029 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:21.790822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.791108 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.791382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.801392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:21.803477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cf61527c-420e-4c0c-b4ca-596560f9263b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:21.821204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:21.822101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:21.827992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:21.827992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:21.827992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:21.827992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:21.827992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1083] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:21.828470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:21.928244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78284712-4639-42a0-8c60-7da8859476e5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:21.956631 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:21.963860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:21.964553 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.964841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.965121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.965851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:21.966164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:21.967542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bdb11-6727-418d-a62f-028f6fbba95b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:21.979492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 549/1084] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:22.519316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf80a-254f-42b3-b80b-5cbbad6ca204 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.519996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf80a-254f-42b3-b80b-5cbbad6ca204 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.519996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf80a-254f-42b3-b80b-5cbbad6ca204 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.522006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1085] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:22.528068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12dcb9-4e1d-4918-91c6-9f8858c0e0c9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.528636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12dcb9-4e1d-4918-91c6-9f8858c0e0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.528636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12dcb9-4e1d-4918-91c6-9f8858c0e0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.546292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2f12dcb9-4e1d-4918-91c6-9f8858c0e0c9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:22.547480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2f12dcb9-4e1d-4918-91c6-9f8858c0e0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:22.622564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 550/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:22.646332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cf80a-254f-42b3-b80b-5cbbad6ca204 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:22.698731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f94b-4dc8-4822-ad46-4f3bca9b0353 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.699019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f94b-4dc8-4822-ad46-4f3bca9b0353 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.699303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f94b-4dc8-4822-ad46-4f3bca9b0353 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.700722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 551/1087] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:22.705788 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.706075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.706357 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.723461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:22.725805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:22.739032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:22.750224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cf80a-254f-42b3-b80b-5cbbad6ca204 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:22.783879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:22.784716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.784998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.785273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.794132 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 552/1088] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:22.795834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:22.804147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:22.805359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:22.806272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1089] 10.222.0.45 () {56 vars in 1120 bytes} [Wed May 25 13:40:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:22.811118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:22.817427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:22.861195 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:22.913650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b7e35-1473-41cb-ab43-1c3a73fd77bf None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:22.917302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a0a6-d5f9-4e6c-9dab-4ff9c02b85f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:22.948051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:22.948757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.948991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.949267 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.950006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:22.950311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:22.951650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:22.951782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:22.952462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:22.952749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:22.953196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:22.960609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 553/1090] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:22.963890 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:22.966211 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:22.971536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:22.972578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:22.973424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1091] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:22.978762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:22.991141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.065607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c63b20-8f2a-4dd0-a7b5-78a99c3b66aa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.079123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.079391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39271fa1-5c78-4b37-ab18-59d98dbeaaad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.101851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.102635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.102920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.103194 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.113576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.113750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.114237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.114668 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.115037 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.115302 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.115788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:23.116136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.117539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.125880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1092] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.132080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:23.133345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:23.134380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:23.135481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:23.135481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:23.135481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:23.135481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:23.135481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 554/1093] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:23.139820 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.202187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.232714 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6163-c092-4ce4-b571-ae85f120777a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.240685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b28cb7-210f-47ce-a314-ce14632dd9b2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.266104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.266963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.267239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.267510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] PATH_INFO: `/v3/users/b6064d3e677549119a0e49e7affbc0fb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.268727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] RBAC: Authorizing `identity:get_user(user_id=b6064d3e677549119a0e49e7affbc0fb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.270163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.272836 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1094] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/users/b6064d3e677549119a0e49e7affbc0fb => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.274690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.275343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.275632 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.275912 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.276663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:23.276974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.278375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.278756 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.290882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 555/1095] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.296371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.322057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.389013 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b224293-0ddd-470c-90b2-ac1586e12902 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.405601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a537414-0a89-465e-9962-91edc2b8861a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.422402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.423160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.423449 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.423730 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.434735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.441009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.441802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.442184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.442354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.450746 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.452292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:40:23.466695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:23.467853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:23.468869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 556/1096] 10.222.0.45 () {56 vars in 1093 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:23.473499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.492317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.493783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.556337 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.573065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ffe4c-600c-45f4-b6b7-b2834e07739c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.592691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1097] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:40:23.598139 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.607359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.608065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.608350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.608626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.609463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:23.609772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.611184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.628900 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 557/1098] 10.222.0.45 () {56 vars in 1133 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.634629 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.642772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.701182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a4b2e-ef45-4b1c-8dc1-998517cf4d8e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.733904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc1734-adf3-4e38-a1c4-18253c5aba89 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.735580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.736486 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.736751 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.737025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.738208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.739697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.741670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1099] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.747416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:23.773726 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.774368 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.774656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.774929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.789029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.845369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.846199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ca402-9306-4136-ae2d-679205d8345b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:23.846860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.857323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:23.882511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:23.883165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:23.883454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:23.883733 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:23.884593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=e23857b9290e4dd9b8ce4f098c168bca, user.id=b6064d3e677549119a0e49e7affbc0fb) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:23.884910 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:23.886341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:23.892152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1100] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/role_assignments?user.id=b6064d3e677549119a0e49e7affbc0fb&scope.project.id=e23857b9290e4dd9b8ce4f098c168bca&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:23.950442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 558/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:40:23.956689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:24.062484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2596-50af-4395-8d9f-87e4c9bd3bbc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:24.098140 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:24.099544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:24.099829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:24.100231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:24.111090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:24.112598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:24.130323 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:24.131622 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:24.132715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:40:24.132715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:24.132715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:40:24.132715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:24.132715 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:24.137284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:24.180607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:24.237157 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182034-7b54-4674-b981-a1f789a0f7bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:24.270223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:24.271428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:24.271710 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:24.272015 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:24.272738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:24.273045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:24.274414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:24.285589 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 559/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:24.291832 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:24.317032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:24.396243 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52dc86-bc79-4bf1-82c5-a36527879808 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:24.429534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:24.430332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:24.430618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:24.430894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:24.437578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:24.439043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:40:24.453853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:24.454761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:24.455645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:24.461644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:24.559014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07713ae-1bf3-4a51-ba68-0713b2767940 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:24.560600 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:24.594219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:24.594805 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:24.595086 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:24.595359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:24.596175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:24.596486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:24.597928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ef0ea-a5c7-4e6c-bd5c-705801553ea2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:24.610734 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 560/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:24.617803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:24.700848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:24.717101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1986-815f-4ff8-a7e6-fdaccced5ddb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:24.751467 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:24.752217 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:24.752506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:24.752783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:24.753612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] RBAC: Adding query filter params (role.id=85ac4472b2064a8d8a1b23a9a2b609f4, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6086ccae239045e5b109e33cf47453a2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:24.753929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:24.755421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094dadc-d07a-4018-ac06-dfde91f03120 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:24.761248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:24 2022] GET /identity/v3/role_assignments?user.id=6086ccae239045e5b109e33cf47453a2&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=85ac4472b2064a8d8a1b23a9a2b609f4 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:26.128290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b46d-c120-49fc-9911-7f3c1ac79ce9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:26.128849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b46d-c120-49fc-9911-7f3c1ac79ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:26.128849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b46d-c120-49fc-9911-7f3c1ac79ce9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:26.130872 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 561/1107] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:26.136310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:26.136753 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:26.136879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:26.154032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:26.155238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:26.219631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:26.244779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30b46d-c120-49fc-9911-7f3c1ac79ce9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:26.345355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30b46d-c120-49fc-9911-7f3c1ac79ce9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:26.379318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:26.380874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:26.381308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:26.381409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:26.382598 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:26.384134 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d5d6c-32fa-4e60-ac6b-3d4bb6ef5d33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:26.385745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 562/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:26.391380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:26.466299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:26.487645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882d6c6-ee5d-4d65-a8f2-1d09d09ca307 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:26.521101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:26.521759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:26.522044 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:26.522320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:26.523034 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:26.524501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d47bc-2eb2-4c37-b6a5-09c147e3d7a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:26.542876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:40:26 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:28.767964 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c870-a16f-47ef-be44-0b002099a442 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:28.768489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c870-a16f-47ef-be44-0b002099a442 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:28.768584 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c870-a16f-47ef-be44-0b002099a442 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:28.769906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 563/1111] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:28.775635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:28.776042 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:28.776221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:28.793346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:28.794535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:28.860014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:28.883507 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65c870-a16f-47ef-be44-0b002099a442 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:28.914103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:28.983881 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65c870-a16f-47ef-be44-0b002099a442 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:29.017058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:29.017899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:29.018231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:29.018460 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] PATH_INFO: `/v3/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:29.019947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] RBAC: Authorizing `identity:get_role(role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:29.022011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:29.023371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 564/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:28 2022] GET /identity/v3/roles/798530c22da24d42b101d2f46d4939bb => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:29.029104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:29.089201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:29.126570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d590dce-cf04-4bc2-8a1d-8c871c551951 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:29.159984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:29.160840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:29.161119 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:29.161430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] PATH_INFO: `/v3/users/04f1d3894791436a95ea46e0e1ac5829` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:29.175220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] RBAC: Authorizing `identity:get_user(user_id=04f1d3894791436a95ea46e0e1ac5829)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:29.176685 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:29.178444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:29 2022] GET /identity/v3/users/04f1d3894791436a95ea46e0e1ac5829 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:29.184226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:29.280893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:29.284895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322dde3-ba34-426c-ab7a-cd78d866b8f1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:29.318972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:29.319612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:29.319895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:29.320202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:29.385911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:29.387561 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:29.479464 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:29.484284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 565/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:29.490528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:29.587152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9bd42-9899-4bdc-9f39-7a8f0d7466a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:29.609558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:29.621898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:29.622747 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:29.623028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:29.623297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:29.624496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:29.625961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879e14e-d75e-4290-8c22-b2b08fdc093f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:29.627625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:29 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:29.634636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:29.731457 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4890-477a-4726-8efe-e75ec2de44a4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:29.764974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:29.765612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:29.765895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:29.766168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:29.766998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=e23857b9290e4dd9b8ce4f098c168bca, user.id=04f1d3894791436a95ea46e0e1ac5829) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:29.767312 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:29.768751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708edd6-c3c8-4ffc-8fcf-c5b0cfbd1ea2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:29.774322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 566/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:29 2022] GET /identity/v3/role_assignments?user.id=04f1d3894791436a95ea46e0e1ac5829&scope.project.id=e23857b9290e4dd9b8ce4f098c168bca&role.id=798530c22da24d42b101d2f46d4939bb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:31.935047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b49d5-8f7f-44a2-b5de-0a79b50adce4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:31.935579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b49d5-8f7f-44a2-b5de-0a79b50adce4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:31.935579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b49d5-8f7f-44a2-b5de-0a79b50adce4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:31.937017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1118] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:31.942088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:31.942471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:31.942692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:31.959427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:31.960640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:32.015838 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:32.026674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 567/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:32.049313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b49d5-8f7f-44a2-b5de-0a79b50adce4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:32.147556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b49d5-8f7f-44a2-b5de-0a79b50adce4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:32.180692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:32.181531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:32.181818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:32.182093 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] PATH_INFO: `/v3/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:32.183547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] RBAC: Authorizing `identity:get_role(role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:32.184942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:32.186339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:32 2022] GET /identity/v3/roles/798530c22da24d42b101d2f46d4939bb => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:32.191862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:32.290644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffeb97-205f-4a74-b60a-16a29dee1135 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:32.323531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:32.324419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:32.324700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:32.324971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] PATH_INFO: `/v3/users/04f1d3894791436a95ea46e0e1ac5829` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:32.326137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] RBAC: Authorizing `identity:get_user(user_id=04f1d3894791436a95ea46e0e1ac5829)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:32.327544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:32.329154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 568/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:32 2022] GET /identity/v3/users/04f1d3894791436a95ea46e0e1ac5829 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:32.335360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:32.392250 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:32.433291 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936b4ce-2fc2-461c-b3d7-380b0c74b224 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:32.465809 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:32.466599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:32.466883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:32.467160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:32.468358 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:32.469796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e9cb0-1057-4052-af08-e658066d37e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:32.471462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:32 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:32.478313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:32.536813 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:32.577223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88ec3-0c67-4742-99a3-1b40e28ef15a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:32.614112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:32.615084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:32.615342 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:32.615615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca/users/04f1d3894791436a95ea46e0e1ac5829/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:32.617821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] RBAC: Authorizing `identity:create_grant(project_id=e23857b9290e4dd9b8ce4f098c168bca, user_id=04f1d3894791436a95ea46e0e1ac5829, role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:32.619242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0399ec-1a50-4b42-a43d-b0087a12f696 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:32.632463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 569/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:40:32 2022] PUT /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca/users/04f1d3894791436a95ea46e0e1ac5829/roles/798530c22da24d42b101d2f46d4939bb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:34.782781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833e481-0d46-4130-b477-2f92e8788716 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:34.783307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833e481-0d46-4130-b477-2f92e8788716 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:34.783307 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833e481-0d46-4130-b477-2f92e8788716 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:34.784731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1124] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:34.790558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:34.790936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:34.791145 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:34.808757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:34.809955 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:34.875551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 570/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:34.900777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833e481-0d46-4130-b477-2f92e8788716 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:34.999370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833e481-0d46-4130-b477-2f92e8788716 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:35.024934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:35.033688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:35.034484 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:35.034767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:35.035043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] PATH_INFO: `/v3/roles/798530c22da24d42b101d2f46d4939bb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:35.036731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] RBAC: Authorizing `identity:get_role(role_id=798530c22da24d42b101d2f46d4939bb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:35.037938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:35.039294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:34 2022] GET /identity/v3/roles/798530c22da24d42b101d2f46d4939bb => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:35.045118 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:35.093783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:35.142356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74457a44-8d78-430d-85bc-797ff07beee2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:35.175452 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:35.176346 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:35.176627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:35.176983 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] PATH_INFO: `/v3/users/04f1d3894791436a95ea46e0e1ac5829` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:35.178160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] RBAC: Authorizing `identity:get_user(user_id=04f1d3894791436a95ea46e0e1ac5829)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:35.179593 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:35.181226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 571/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:35 2022] GET /identity/v3/users/04f1d3894791436a95ea46e0e1ac5829 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:35.186847 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:35.284645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4f56-85c9-4516-960e-ef447004069d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:35.318101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:35.318745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:35.319032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:35.319311 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:35.381678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:35.389731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:35.391189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:35.495728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:35.501577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:35.599995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693efd-9f96-474e-8130-da088948c2dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:35.617614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:35.635616 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:35.636482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:35.636768 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:35.637036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] PATH_INFO: `/v3/projects/e23857b9290e4dd9b8ce4f098c168bca` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:35.638193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] RBAC: Authorizing `identity:get_project(project_id=e23857b9290e4dd9b8ce4f098c168bca)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:35.639595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b2720-3287-4272-86cd-df5989525e11 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:35.641276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 572/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:35 2022] GET /identity/v3/projects/e23857b9290e4dd9b8ce4f098c168bca => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:35.647298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:35.737309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:35.744896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34914b70-965f-419d-9812-b6c7389cb68f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:35.778114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:35.778756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:35.779041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:35.779319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:35.780172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] RBAC: Adding query filter params (role.id=798530c22da24d42b101d2f46d4939bb, scope.project.id=e23857b9290e4dd9b8ce4f098c168bca, user.id=04f1d3894791436a95ea46e0e1ac5829) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:35.780497 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:35.781935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615700c-3eb8-4c8f-b265-9341950fd839 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:35.788218 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:35 2022] GET /identity/v3/role_assignments?user.id=04f1d3894791436a95ea46e0e1ac5829&scope.project.id=e23857b9290e4dd9b8ce4f098c168bca&role.id=798530c22da24d42b101d2f46d4939bb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:38.001560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c1a-5551-4d45-8ccd-747d5414756a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:38.002107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c1a-5551-4d45-8ccd-747d5414756a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:38.002107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c1a-5551-4d45-8ccd-747d5414756a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:38.003466 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 573/1131] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:38.009362 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:38.009748 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:38.009934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:38.027002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:38.028227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:38.093838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:38.116892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35c1a-5551-4d45-8ccd-747d5414756a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:38.216421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35c1a-5551-4d45-8ccd-747d5414756a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:38.249251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:38.250029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:38.250314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:38.250588 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:38.251816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:38.253216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10cef-94a6-454d-b03e-0d9c2b1466b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:38.254802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 574/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:38.261446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:38.347109 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:38.361383 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c03659-f102-4005-b370-74b78ee9f5db None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:38.395470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:38.396501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:38.396787 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:38.397088 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:38.397807 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:38.399248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f60ff8-addd-4b63-b41f-dff4800de3b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:38.423255 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:40:38 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:40.640412 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd89704-3c4e-4b11-888b-1b8ab3c136e5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:40.640926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd89704-3c4e-4b11-888b-1b8ab3c136e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:40.641023 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd89704-3c4e-4b11-888b-1b8ab3c136e5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:40.642324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 575/1135] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:40.647871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:40.648338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:40.648463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:40.664990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:40.666176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:40.730981 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:40.756524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd89704-3c4e-4b11-888b-1b8ab3c136e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:40.811627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:40.858580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd89704-3c4e-4b11-888b-1b8ab3c136e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:40.893207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:40.893927 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:40.894213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:40.894490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:40.895706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:40.897243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba59173-74e1-42ed-af83-0fba451ff520 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:40.898844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 576/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:40:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:40.904620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:40.970870 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:41.002461 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480aa2c6-7004-49c1-bf66-ce91219b1ab6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:41.035504 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:41.036731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:41.037105 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:41.037386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:41.038101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:41.039534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75adcf36-9247-44d2-9a5f-6232450633d4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:41.058410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:40:40 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:43.263799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6c5c6-7b96-41d6-9718-1a53d236fb7a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.264397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6c5c6-7b96-41d6-9718-1a53d236fb7a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.264397 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6c5c6-7b96-41d6-9718-1a53d236fb7a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:43.265761 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 577/1139] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:43.271436 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.271806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.272054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:43.289103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:43.290282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:43.354618 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:43.378300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6c5c6-7b96-41d6-9718-1a53d236fb7a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:43.479197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6c5c6-7b96-41d6-9718-1a53d236fb7a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:43.502116 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:43.514133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:43.514854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.515139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.515417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:43.526079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:43.533695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:43.534780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:43.536010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 578/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:43.541512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:43.595348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:43.639445 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c48b6b-adf7-465e-afdd-e467fe542191 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:43.672868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:43.673528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.673813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.674090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:43.674838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:43.675150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:43.676560 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:43.684479 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:43.690234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:43.788761 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c48d9-1d69-4dd3-94fd-2603d8a73ec7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:43.820929 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:43.821766 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.822048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.822323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] PATH_INFO: `/v3/users/536ba054c9cc42458d43e7cc6bfa4196` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:43.835390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] RBAC: Authorizing `identity:get_user(user_id=536ba054c9cc42458d43e7cc6bfa4196)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:43.836845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:43.838491 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 579/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:43 2022] GET /identity/v3/users/536ba054c9cc42458d43e7cc6bfa4196 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:43.844092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:43.940743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd4a9-2e4e-43d1-b227-b3920ccbbafe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:43.957515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:43.975057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:43.975697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:43.976017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:43.976299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:44.041707 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:44.043087 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:44.138155 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:44.143865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:44.185377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:44.242240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b01e0-8d52-49a2-941b-356fb524c77b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:44.274960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:44.275799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:44.276109 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:44.276383 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] PATH_INFO: `/v3/projects/b3474d8028a7447eae802c7f79b85cef` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:44.277551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] RBAC: Authorizing `identity:get_project(project_id=b3474d8028a7447eae802c7f79b85cef)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:44.278962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740e6a-2e9e-402f-b2b2-c5bbef860349 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:44.280662 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 580/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:44 2022] GET /identity/v3/projects/b3474d8028a7447eae802c7f79b85cef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:44.286624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:44.304900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:44.384838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd3dfa-6ac3-4d43-b623-d591272b6ff6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:44.416979 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:44.417625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:44.417912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:44.418189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:44.419014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=b3474d8028a7447eae802c7f79b85cef, user.id=536ba054c9cc42458d43e7cc6bfa4196) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:44.419330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:44.420796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9fc05-8f38-4882-bc34-a65807a6cac6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:44.426093 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:44 2022] GET /identity/v3/role_assignments?user.id=536ba054c9cc42458d43e7cc6bfa4196&scope.project.id=b3474d8028a7447eae802c7f79b85cef&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:46.612859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0be981-417c-46c4-a2e2-165062f78a3d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:46.613427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0be981-417c-46c4-a2e2-165062f78a3d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:46.613427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0be981-417c-46c4-a2e2-165062f78a3d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:46.615774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 581/1147] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:46.620921 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:46.621340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:46.621698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:46.638871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:46.640165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:46.705969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:46.730018 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0be981-417c-46c4-a2e2-165062f78a3d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:46.828671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0be981-417c-46c4-a2e2-165062f78a3d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:46.837605 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:46.862068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:46.862842 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:46.863173 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:46.863394 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:46.875119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:46.883239 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:46.884388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:46.885414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 582/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:46.890498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:46.904273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:46.988639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27087cc1-c53c-4e30-8644-2a29a3d229ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:47.021842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:47.022506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:47.022793 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:47.023071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:47.023818 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:47.024158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:47.025535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:47.033427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:47.040057 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:47.141164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cb4e0-dd36-4d54-9d14-0212568dbba3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:47.174206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:47.175063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:47.175347 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:47.175623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] PATH_INFO: `/v3/users/536ba054c9cc42458d43e7cc6bfa4196` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:47.177487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=536ba054c9cc42458d43e7cc6bfa4196)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:47.178878 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:47.180513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 583/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:47 2022] GET /identity/v3/users/536ba054c9cc42458d43e7cc6bfa4196 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:47.186196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:47.259153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:47.284170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd46ef2-bd7f-4041-80e2-e222a1a614f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:47.316896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:47.317895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:47.318732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:47.319008 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] PATH_INFO: `/v3/projects/b3474d8028a7447eae802c7f79b85cef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:47.320213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] RBAC: Authorizing `identity:get_project(project_id=b3474d8028a7447eae802c7f79b85cef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:47.321644 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aceefb-6877-4737-92ee-c81ad7bf21ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:47.323317 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:47 2022] GET /identity/v3/projects/b3474d8028a7447eae802c7f79b85cef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:47.329105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:47.365411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:47.428769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660e03-528e-4b20-a3ab-a46808b19ba7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:47.461724 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:47.462596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:47.462880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:47.463155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] PATH_INFO: `/v3/projects/b3474d8028a7447eae802c7f79b85cef/users/536ba054c9cc42458d43e7cc6bfa4196/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:47.465091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] RBAC: Authorizing `identity:create_grant(project_id=b3474d8028a7447eae802c7f79b85cef, user_id=536ba054c9cc42458d43e7cc6bfa4196, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:47.466513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c6b2-d24b-4025-ae7b-1e58b4eda5fa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:47.480096 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 584/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:40:47 2022] PUT /identity/v3/projects/b3474d8028a7447eae802c7f79b85cef/users/536ba054c9cc42458d43e7cc6bfa4196/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:49.642477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e74735-bf31-4910-bcff-becc4709a271 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:49.642986 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e74735-bf31-4910-bcff-becc4709a271 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:49.643081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e74735-bf31-4910-bcff-becc4709a271 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:49.644469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1154] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:49.649635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:49.650011 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:49.650196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:49.666801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:49.668001 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:49.733722 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 585/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:49.757517 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e74735-bf31-4910-bcff-becc4709a271 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:49.856325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e74735-bf31-4910-bcff-becc4709a271 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:49.889211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:49.889988 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:49.890273 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:49.890548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:49.901245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:49.908874 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:49.909995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:40:49.911081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:40:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:49.916388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:49.942142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:50.015158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10082-55ed-4fda-9ae3-2e70dcb5898d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:50.048030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:50.048690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:50.048977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:50.049257 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:50.050006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:50.050314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:50.051675 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:50.059603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 586/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:50.065294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:50.068862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:50.165252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accac816-07ed-47ce-8d78-4c260a1a0e77 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:50.197384 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:50.198236 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:50.198516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:50.198788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] PATH_INFO: `/v3/users/536ba054c9cc42458d43e7cc6bfa4196` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:50.199990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] RBAC: Authorizing `identity:get_user(user_id=536ba054c9cc42458d43e7cc6bfa4196)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:50.201437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:50.203038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:40:50 2022] GET /identity/v3/users/536ba054c9cc42458d43e7cc6bfa4196 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:50.209335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:50.294668 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:50.307408 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc05e9-b6f2-450e-b2ec-b2fd73d8cf14 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:50.340272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:50.340907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:50.341190 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:50.341465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:50.406913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:50.408404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:50.490678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:50.502941 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 587/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:50.509642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:50.576538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:50.607406 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aae80-6056-47b6-bded-00863a5519c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:50.641142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:50.641991 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:50.642275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:50.642554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] PATH_INFO: `/v3/projects/b3474d8028a7447eae802c7f79b85cef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:50.643745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] RBAC: Authorizing `identity:get_project(project_id=b3474d8028a7447eae802c7f79b85cef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:50.645244 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661d8ca-1f82-4edf-94d8-35b19b4bd928 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:50.646952 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:40:50 2022] GET /identity/v3/projects/b3474d8028a7447eae802c7f79b85cef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:50.653123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:50.750617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab7b57-0b03-432e-a371-8655cd744feb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:50.783026 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:50.783664 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:50.783973 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:50.784258 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:50.785087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=b3474d8028a7447eae802c7f79b85cef, user.id=536ba054c9cc42458d43e7cc6bfa4196) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:50.785404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:50.787435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd80dd-b4a9-4852-adfa-e6362e17d2ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:50.793190 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 588/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:40:50 2022] GET /identity/v3/role_assignments?user.id=536ba054c9cc42458d43e7cc6bfa4196&scope.project.id=b3474d8028a7447eae802c7f79b85cef&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:53.026889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ac78-b9fd-4203-b528-081c463b1e43 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:53.027454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ac78-b9fd-4203-b528-081c463b1e43 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:53.027454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ac78-b9fd-4203-b528-081c463b1e43 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:53.028840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1162] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:53.036998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:53.036998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:53.037857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:53.054926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:53.056160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:53.104403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:53.125576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 589/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:53.150285 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ac78-b9fd-4203-b528-081c463b1e43 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:53.266752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ac78-b9fd-4203-b528-081c463b1e43 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:53.323415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:53.324073 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:53.324360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:53.325686 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:53.335370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:53.415150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:53.416637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:53.523040 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:40:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:53.529133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:53.634547 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723313d-07fd-4d1f-a6c4-4ee5dbcc8742 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:53.667897 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:53.668719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:53.668999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:53.669273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:53.675775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:53.677819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cb86966d-298e-4039-b87f-ada5ab744c1f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:53.691986 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:53.693698 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:40:53.694920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 590/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:40:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:53.699199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:53.738333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:53.796703 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e2b60-b16a-449b-a32b-2004c6d8cd33 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:53.830001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:53.830653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:53.830934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:53.831208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:53.832463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:53.832816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:53.834302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4e521-b27d-4f37-b31c-af87deeb85f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:53.841698 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Wed May 25 13:40:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:56.016914 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346052c-3bc0-4810-960f-2cbf76ecb096 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:56.017458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346052c-3bc0-4810-960f-2cbf76ecb096 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:56.017458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346052c-3bc0-4810-960f-2cbf76ecb096 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:56.018841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 591/1167] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:56.024805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21e79e-e35f-401f-8ea5-9054d82f9b02 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:56.025232 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21e79e-e35f-401f-8ea5-9054d82f9b02 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:56.025370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21e79e-e35f-401f-8ea5-9054d82f9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:56.042038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-af21e79e-e35f-401f-8ea5-9054d82f9b02 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:56.043231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af21e79e-e35f-401f-8ea5-9054d82f9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:56.109145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:56.133175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346052c-3bc0-4810-960f-2cbf76ecb096 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:56.177557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:56.234339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346052c-3bc0-4810-960f-2cbf76ecb096 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:56.267481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:56.268093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:56.268379 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:56.268653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:56.269406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:56.271371 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c7afe-49f2-45a0-83ce-7f2b1fabb0a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:56.285432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 592/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Wed May 25 13:40:56 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:58.481388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd9f-fbfb-40c8-b21d-3c340bcaaf7a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:58.481924 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd9f-fbfb-40c8-b21d-3c340bcaaf7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:58.482018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54dd9f-fbfb-40c8-b21d-3c340bcaaf7a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:58.483351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1170] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:58.488697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:58.489133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:58.489264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:58.505991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:58.507179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:58.572970 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 593/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:40:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:58.596148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54dd9f-fbfb-40c8-b21d-3c340bcaaf7a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:58.627208 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:58.696535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54dd9f-fbfb-40c8-b21d-3c340bcaaf7a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:58.729744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:58.730586 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:58.730869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:58.731145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] PATH_INFO: `/v3/domains/44589701c5724a68b2623e4a255fa947` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:58.741356 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=44589701c5724a68b2623e4a255fa947)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:58.742805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ecbcf-d4aa-4108-aa38-2db1726ac393 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:58.744444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Wed May 25 13:40:58 2022] GET /identity/v3/domains/44589701c5724a68b2623e4a255fa947 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:58.750169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:58.780152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:58.848248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853e648-5c56-4e94-ab88-d661736cf62f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:58.881966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:58.882559 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:58.882846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:58.883126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:58.883847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:58.885296 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00810e-80fd-4fba-b9da-85515dc04fc2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:58.910353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 594/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:40:58 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:41:01.118708 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd97a69-655a-4e66-9660-f348f75d4992 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:01.119238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd97a69-655a-4e66-9660-f348f75d4992 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:01.119238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd97a69-655a-4e66-9660-f348f75d4992 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:01.120651 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1174] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:01.125783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:01.126153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:01.126350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:01.143669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:01.144889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:01.211062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 595/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:41:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:01.236149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd97a69-655a-4e66-9660-f348f75d4992 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:01.326431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:01.337099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd97a69-655a-4e66-9660-f348f75d4992 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:01.370263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:01.371062 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:01.371344 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:01.371615 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] PATH_INFO: `/v3/domains/44589701c5724a68b2623e4a255fa947` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:01.372823 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=44589701c5724a68b2623e4a255fa947)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:01.374254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955e215-9551-45a0-a822-7167bce12de4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:01.375859 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Wed May 25 13:41:01 2022] GET /identity/v3/domains/44589701c5724a68b2623e4a255fa947 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:01.382127 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:01.397472 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:01.481073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96180fdb-f169-4138-9b80-b8e688559d33 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:01.514674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:01.515330 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:01.515614 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:01.515892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:01.516639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:01.518050 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5eaf9-076a-4854-86b5-c8bb500dd268 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:01.538639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 596/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:41:01 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:41:03.773279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bdab-372f-445d-a3ec-90a9bb627be7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:03.773828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bdab-372f-445d-a3ec-90a9bb627be7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:03.773828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bdab-372f-445d-a3ec-90a9bb627be7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:03.775209 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1178] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:03.780566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:03.780987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:03.781131 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:03.797651 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:03.798815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:03.860871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:03.866741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 597/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:41:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:03.890038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036bdab-372f-445d-a3ec-90a9bb627be7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:03.987858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036bdab-372f-445d-a3ec-90a9bb627be7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.021601 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.022378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.022664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.022943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.032846 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:04.041310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:04.042432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:41:04.043485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:04.048581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:04.143911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c3e63-17ed-44a4-a14f-948a7c7c1090 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.177158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.177811 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.178097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.178373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.179460 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:04.179772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:04.181155 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:04.188993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 598/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:04.195247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:04.217552 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:04.293794 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273997c-ea84-430c-880b-89a115ca9671 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.326252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.327106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.327389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.327663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] PATH_INFO: `/v3/users/e534dff76de947f6b1c1260404a2c064` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.341397 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] RBAC: Authorizing `identity:get_user(user_id=e534dff76de947f6b1c1260404a2c064)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:04.342832 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:04.344536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:41:04 2022] GET /identity/v3/users/e534dff76de947f6b1c1260404a2c064 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:04.350313 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:04.410804 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:04.449307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f34a95-0cfa-4e64-8a17-84b0f0f0a68a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.481867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.482495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.482776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.483047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.547073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:04.548453 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:04.605277 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:04.643471 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 599/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:41:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:41:04.649934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:04.725474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:04.746604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c61ee5-9ae0-467c-b36d-20a0a1f6b624 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.779750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.780636 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.780922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.781200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] PATH_INFO: `/v3/projects/7fdabd9e16a945468791b289a38d6600` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.782367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] RBAC: Authorizing `identity:get_project(project_id=7fdabd9e16a945468791b289a38d6600)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:04.783797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7b066-1173-4a06-9dd3-442616d73a50 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:04.785510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:41:04 2022] GET /identity/v3/projects/7fdabd9e16a945468791b289a38d6600 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:04.792669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:04.889393 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8971-6be0-4c43-a066-dc72c88673e9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:04.922829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:04.923463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:04.923741 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:04.924039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:04.924858 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=7fdabd9e16a945468791b289a38d6600, user.id=e534dff76de947f6b1c1260404a2c064) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:04.925170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:04.926581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77511a-dc80-4666-be85-ca71be5c7dd6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:04.932481 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 600/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:41:04 2022] GET /identity/v3/role_assignments?user.id=e534dff76de947f6b1c1260404a2c064&scope.project.id=7fdabd9e16a945468791b289a38d6600&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:07.079554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156414a-240a-44e9-a684-4751a52770b0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.080124 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156414a-240a-44e9-a684-4751a52770b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.080124 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156414a-240a-44e9-a684-4751a52770b0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.081513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1186] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:07.087375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed15-902e-492e-9103-c81c73b3599b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.087769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed15-902e-492e-9103-c81c73b3599b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.087954 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aed15-902e-492e-9103-c81c73b3599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.105895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-785aed15-902e-492e-9103-c81c73b3599b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:07.107062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-785aed15-902e-492e-9103-c81c73b3599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:07.126469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:07.172351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 601/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:07.194938 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156414a-240a-44e9-a684-4751a52770b0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:07.294464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156414a-240a-44e9-a684-4751a52770b0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:07.327116 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:07.327891 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.328210 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.328490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.338888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:07.346540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:07.347518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:41:07.348381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:07.353988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aed15-902e-492e-9103-c81c73b3599b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:07.449146 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aed15-902e-492e-9103-c81c73b3599b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:07.482618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:07.483273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.483556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.483832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.484604 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:07.484913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:07.486279 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:07.494732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 602/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:07.500603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:07.511007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:07.601352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e433-e939-47c5-b935-016f7645b334 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:07.635100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:07.635996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.636283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.636558 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] PATH_INFO: `/v3/users/e534dff76de947f6b1c1260404a2c064` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.637734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] RBAC: Authorizing `identity:get_user(user_id=e534dff76de947f6b1c1260404a2c064)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:07.639684 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:07.641348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:41:07 2022] GET /identity/v3/users/e534dff76de947f6b1c1260404a2c064 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:07.647252 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:07.666374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:07.745633 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f5813-7833-4940-8e68-05da4898cf30 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:07.778195 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:07.779053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.779339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.779615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] PATH_INFO: `/v3/projects/7fdabd9e16a945468791b289a38d6600` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.780826 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] RBAC: Authorizing `identity:get_project(project_id=7fdabd9e16a945468791b289a38d6600)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:07.782247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38138c1-a2f7-4397-807d-1b0f2e1da373 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:07.783943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 603/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:41:07 2022] GET /identity/v3/projects/7fdabd9e16a945468791b289a38d6600 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:07.789773 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:07.862856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:07.888047 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d27f46-9bb5-48c2-a040-286dd5d5cd26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:07.920452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:07.921322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:07.921607 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:07.921881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] PATH_INFO: `/v3/projects/7fdabd9e16a945468791b289a38d6600/users/e534dff76de947f6b1c1260404a2c064/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:07.923838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] RBAC: Authorizing `identity:create_grant(project_id=7fdabd9e16a945468791b289a38d6600, user_id=e534dff76de947f6b1c1260404a2c064, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:07.925297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d966692-cf88-43ad-9fe2-6ac4730286ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:07.937321 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:41:07 2022] PUT /identity/v3/projects/7fdabd9e16a945468791b289a38d6600/users/e534dff76de947f6b1c1260404a2c064/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:41:10.096527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd785bb-a164-453d-9839-d62780113dbc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.097060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd785bb-a164-453d-9839-d62780113dbc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.097060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd785bb-a164-453d-9839-d62780113dbc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.098480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 604/1193] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:10.104227 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.104629 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.104800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.121948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:10.123131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:10.188643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:41:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:10.211577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd785bb-a164-453d-9839-d62780113dbc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:10.296731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:10.313184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd785bb-a164-453d-9839-d62780113dbc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:10.345719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:10.346518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.346859 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.347088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.358498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:10.366791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:10.367886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:41:10.369047 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 605/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:10.373866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:10.434151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:10.471463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22b99-3ef1-43a7-9811-a9d9c8c178bc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:10.503939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:10.504605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.504885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.506134 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.506880 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:10.507189 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:10.508578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:10.516418 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:10.522245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:10.620911 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713045a-5df2-4987-bf33-23bd97741e1c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:10.653966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:10.654814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.655098 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.655374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] PATH_INFO: `/v3/users/e534dff76de947f6b1c1260404a2c064` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.656566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] RBAC: Authorizing `identity:get_user(user_id=e534dff76de947f6b1c1260404a2c064)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:10.657995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:10.659590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 606/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed May 25 13:41:10 2022] GET /identity/v3/users/e534dff76de947f6b1c1260404a2c064 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:10.665195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:10.762419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0e982-72f1-49ec-b3ae-b2490e1b5fa3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:10.786660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:10.795470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:10.796141 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:10.796427 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:10.796705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:10.862408 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:10.863858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:10.959575 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:41:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:41:10.965302 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:10.981450 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:11.064513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ce3-1177-4bfe-914a-c1786e33ef59 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:11.097831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:11.098685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:11.099191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:11.099512 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] PATH_INFO: `/v3/projects/7fdabd9e16a945468791b289a38d6600` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:11.100721 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] RBAC: Authorizing `identity:get_project(project_id=7fdabd9e16a945468791b289a38d6600)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:11.102419 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333810f-b0d9-4faa-951c-97eecda0abaf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:11.104114 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 607/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Wed May 25 13:41:10 2022] GET /identity/v3/projects/7fdabd9e16a945468791b289a38d6600 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:11.110246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:11.137006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:11.207811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c5d3c-20df-4d4b-a6d4-318b2a59a223 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:11.241697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:11.242282 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:11.242567 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:11.242844 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:11.243666 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=7fdabd9e16a945468791b289a38d6600, user.id=e534dff76de947f6b1c1260404a2c064) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:11.244006 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:11.245442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32b2a1-6376-4007-8d41-7d9917122fd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:11.251149 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:41:11 2022] GET /identity/v3/role_assignments?user.id=e534dff76de947f6b1c1260404a2c064&scope.project.id=7fdabd9e16a945468791b289a38d6600&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:21.093709 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=59 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:21.248521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=55 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:11.642509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc432a5-b68d-4e93-b9de-e88189b07080 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:11.643251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc432a5-b68d-4e93-b9de-e88189b07080 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:11.643251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc432a5-b68d-4e93-b9de-e88189b07080 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:11.644891 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 608/1201] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:11.651744 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:11.652363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:11.652363 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:11.687276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:11.688989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:11.783727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:11.814044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc432a5-b68d-4e93-b9de-e88189b07080 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:11.970794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc432a5-b68d-4e93-b9de-e88189b07080 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.032445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.034044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.034044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.034044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.048984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.050830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:12.077742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:12.078839 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:12.079806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:12.079806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:12.079806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:12.079806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:12.079806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 609/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:12.085790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:12.223826 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010cc6e-3e23-4081-86c5-e6045398f1fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.257453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:12.264719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.265620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.265908 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.266496 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.267481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:12.268008 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.270526 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d7e5-a913-48db-8f72-90997cc5ca55 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.286470 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:12.293318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:12.365988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:12.441094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c44cd3-d37e-4324-b4a9-2e22d780f100 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.500033 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.501900 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.502251 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.502531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.503262 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.504854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b8d31-d38f-4f92-9eb3-83bc71f00603 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.539484 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 610/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Wed May 25 13:42:12 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:42:14.985244 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc91744-0242-4b36-be9a-5c70f898af5a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:14.985813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc91744-0242-4b36-be9a-5c70f898af5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:14.985813 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc91744-0242-4b36-be9a-5c70f898af5a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:14.987473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1206] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:14.993770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:14.994206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:14.994345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.012517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:15.013732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:15.088893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 611/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:15.114294 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc91744-0242-4b36-be9a-5c70f898af5a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.247081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc91744-0242-4b36-be9a-5c70f898af5a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.285811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.286598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.287555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.287848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.301055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:15.313438 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:15.314443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:15.315326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:15.322350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.370190 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:15.439121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a9cd3-51f9-4f98-8f53-c5bd8719642b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.482870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.483645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.483976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.484358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.485115 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:15.485426 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:15.487194 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:15.497188 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 612/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:15.503638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.566112 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:15.626003 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d11cbf-408e-4ae5-94ba-ee1bb261f85d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.665505 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.666444 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.666833 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.667219 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.684551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:15.686216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:15.705163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:15.707399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:15.708727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:15.708727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:15.708727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:15.708727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:15.708727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:42:15.714084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.827751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:15.832784 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93eb7-b80a-4560-9e35-99088303b2c8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.872480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.873147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.873467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.873747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.874469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:15.874781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:15.876255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:15.892220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 613/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:15.895229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:16.020491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016f4d1-aaf2-476c-b261-76bc410e0819 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:16.022194 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:16.066094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:16.067064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:16.068137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:16.068137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:16.080198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:16.080354 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:16.097603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:16.098658 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:16.100693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:16.104866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:16.224796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca0908-f7e7-4357-adf9-935aaae1328b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:16.264750 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:16.265517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:16.265825 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:16.266140 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:16.267014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:16.267326 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:16.269264 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:16.290533 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 614/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:16.296182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:16.425102 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a810ae-5f9a-43d0-a56f-fde9f197c006 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:16.463728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:16.464402 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:16.464731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:16.465027 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:16.499571 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:16.545580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:16.547071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:16.663835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:42:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:42:16.669759 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:16.690156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:16.783203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3c27-359d-428e-8dd9-13f5efd1cb4e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:16.822575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:16.823475 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:16.823857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:16.825247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:16.838004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:16.839536 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:16.857930 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:16.859051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:16.865366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:16.865366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:16.865366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:16.865366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:16.865366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 615/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:16.866359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:16.939822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:16.988342 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbadb3-210a-4d76-9f16-7e2083e43960 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:17.027691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:17.028515 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:17.028806 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:17.029086 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:17.029819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:17.030130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:17.031617 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:17.045319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:17.051068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:17.140848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:17.175712 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d9032-41cd-4870-a775-700acd33e9a9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:17.216463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:17.217290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:17.217577 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:17.217853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:17.225877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:17.227378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:17.242245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:17.243341 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:17.244487 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 616/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:17.249570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:17.371322 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e55ff0-2f5e-44b0-9bd9-298f2ba4db2e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:17.394530 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:17.408172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:17.408945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:17.409233 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:17.409512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:17.410557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:17.410919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:17.413246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8261e-829b-42c2-b9fa-c19f99ee0cbf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:17.433032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:17.434406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:17.563514 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801cbaa-5131-445d-bf4b-6682bcf647d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:17.601029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:17.611014 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:17.611912 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:17.612286 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:17.612700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:17.614182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6f6d6866832f456d84536d3cc45e512c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:17.614618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:17.616359 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd980b2c-cd9c-461b-9913-a8f6d9959f4e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:17.631952 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 617/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:42:17 2022] GET /identity/v3/role_assignments?user.id=6f6d6866832f456d84536d3cc45e512c&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:19.924319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd092ef-d967-416f-9ea1-ccff048a8f24 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:19.924877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd092ef-d967-416f-9ea1-ccff048a8f24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:19.924877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd092ef-d967-416f-9ea1-ccff048a8f24 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:19.926376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1220] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:19.932676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:19.933105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:19.933281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:19.952971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:19.954496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:20.038121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 618/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:20.063699 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd092ef-d967-416f-9ea1-ccff048a8f24 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:20.204978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd092ef-d967-416f-9ea1-ccff048a8f24 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:20.244248 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:20.245480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:20.245871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:20.246185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:20.259655 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:20.267754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:20.268973 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:20.276637 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:20.277458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:20.384504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14cac7-4249-43ee-bc71-9210a1e32793 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:20.409540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:20.421749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:20.422418 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:20.422704 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:20.422982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:20.423732 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:20.424079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:20.425435 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:20.434244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 619/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:20.440328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:20.492978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:20.555577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1b5cd-5419-4509-9ea7-80a3b9740907 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:20.589431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:20.590238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:20.590524 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:20.590799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:20.602722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:20.604510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:20.623643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:20.624985 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:20.625885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:20.625885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:20.625885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:20.625885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:20.625885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:20.631945 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:20.750697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85cae5-6f1b-4536-a6a3-96b6c2eaa6c7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:20.786862 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:20.787557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:20.787833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:20.788135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:20.788907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:20.789156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:20.790502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:20.802499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 620/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:20.808745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:20.909895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:20.918330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41f6e-28dd-4461-93db-593d6244cd5c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:20.957286 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:20.958128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:20.958377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:20.959338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:20.967121 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:20.968978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:20.984128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:20.985338 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:20.986292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:20.991411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.043869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:21.111888 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4b6a-d982-468b-aec4-e7b660899cf7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:21.158305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:21.158988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:21.159375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:21.159772 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:21.160705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:21.161070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:21.163454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:21.182432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 621/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:21.188060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.302741 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b820b2-9864-4a98-9561-862954919aaa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:21.342536 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:21.343330 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:21.343648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:21.343993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:21.359868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:21.361615 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:21.382030 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:21.383226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:21.384547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:21.384547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:21.384547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:21.384547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:21.384547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:21.389401 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.501959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68da528-5332-4026-b560-e3594dde60e9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:21.504502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:21.540035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:21.540695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:21.540974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:21.541247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:21.542006 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:21.542309 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:21.543663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:21.557546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 622/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:42:21.563705 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.574548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:21.674222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3322aa-88c5-43ad-9826-a03debaf96bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:21.709548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:21.710471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:21.710757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:21.711033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:21.718556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:21.720202 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:21.736670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:21.737759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:21.738975 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:42:21.744597 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.861581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576403-81b7-4c80-840b-92f66f73a783 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:21.899524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:21.900255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:21.900550 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:21.900827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:21.901687 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:21.902038 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:21.903695 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88910e80-1305-45d6-ae21-14cfda22e43d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:21.917609 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 623/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:21.923984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:21.997415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:22.035731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ef109-00fd-408e-8492-862ab333f881 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:22.072201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:22.073085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:22.073367 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:22.073635 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6f6d6866832f456d84536d3cc45e512c/roles/3bbb70ac844e475ea07d282e37dd3243` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:22.089380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=6f6d6866832f456d84536d3cc45e512c, role_id=3bbb70ac844e475ea07d282e37dd3243)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:22.090999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea795da8-5d7b-4ed7-8dd6-b510780b8033 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:22.106834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:42:21 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6f6d6866832f456d84536d3cc45e512c/roles/3bbb70ac844e475ea07d282e37dd3243 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:42:24.505366 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10feb2c-9f74-4bd5-bb68-0ab33d202da8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:24.507857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10feb2c-9f74-4bd5-bb68-0ab33d202da8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:24.507857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10feb2c-9f74-4bd5-bb68-0ab33d202da8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:24.508449 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 624/1233] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:42:24.514399 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:24.514869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:24.515192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:24.534174 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:24.535464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:24.610721 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:24.637396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10feb2c-9f74-4bd5-bb68-0ab33d202da8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:24.666647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:24.758396 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10feb2c-9f74-4bd5-bb68-0ab33d202da8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:24.796207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:24.797064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:24.797390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:24.797680 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:24.810392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:24.819482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:24.820492 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:24.821373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 625/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Wed May 25 13:42:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:42:24.826909 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:24.882298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:24.940166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddf706-9b00-44d7-9b1d-a7d71ab60cc5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:24.979166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:24.979915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:24.980269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:24.980595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:24.981494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:24.981918 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:24.983458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:24.992746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Wed May 25 13:42:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:24.998782 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:25.096407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:25.116877 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7380a7-355c-4157-97b0-2f0381fd75c1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:25.159864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:25.160763 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:25.161051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:25.161405 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:25.175404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:25.177130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:25.198049 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:25.199022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:25.200218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:25.200218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:25.200218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:25.200218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:25.200218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 626/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:25.205546 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:25.328593 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44837dae-de18-4dd4-bef5-d618a74d1280 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:25.339340 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:25.376531 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:25.377697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:25.378230 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:25.378710 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:25.380158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:25.381316 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:25.383680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:25.404389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:25.410501 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:25.517890 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ca576-4bcd-4fbd-bd1e-7d619f4cec14 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:25.555322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:25.556182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:25.556454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:25.556902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:25.560174 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:25.566186 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:25.567982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:25.586202 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:25.587398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:25.588534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 627/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:42:25.593551 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:25.711722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436865f-b1e7-4d48-92c8-6e8bf3886cd7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:25.753962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:25.754803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:25.755092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:25.755375 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:25.757109 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:25.757494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:25.759146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:25.779643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:25.786495 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:25.903391 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd35f-eab7-4af4-aa9f-2af0790e495a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:25.941760 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:25.942523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:25.942857 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:25.943902 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:26.012969 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:26.019890 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:26.021461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:26.140285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 628/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:42:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:42:26.148638 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:26.187883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:26.261483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c3b14-ddf6-4940-9e9c-daa0fe341c12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:26.298423 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:26.299136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:26.299417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:26.299692 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:26.312667 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:26.314249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:26.335643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:26.336875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:26.337964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:26.337964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:26.337964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:26.337964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:26.337964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:26.343289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:26.454207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:26.461755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbf37c-8675-422f-b817-7e7de76a1b99 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:26.503184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:26.503875 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:26.504222 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:26.504524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:26.505339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:26.506353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:26.507967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:26.521250 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 629/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:42:26.527873 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:26.621237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:26.645974 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa135092-c73e-40be-a4fd-c471e110678d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:26.689838 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388078-789a-4777-b41a-1597b258816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:26.691018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388078-789a-4777-b41a-1597b258816c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:26.691403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388078-789a-4777-b41a-1597b258816c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:26.691740 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388078-789a-4777-b41a-1597b258816c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:26.701434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53388078-789a-4777-b41a-1597b258816c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:26.703389 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53388078-789a-4777-b41a-1597b258816c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-53388078-789a-4777-b41a-1597b258816c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:26.719895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:26.721156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:26.722474 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:26.728762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:26.847721 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464f00-a380-4ee8-a286-2743742f6a26 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:26.886736 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:26.887413 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:26.887745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:26.888103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:26.889144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:26.889478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:26.891090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcb100-cc0a-4721-9f6a-504b0b4c23b5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:26.905102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 630/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:26.912527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388078-789a-4777-b41a-1597b258816c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:27.026078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388078-789a-4777-b41a-1597b258816c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:27.078426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:27.079247 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:27.079626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:27.080043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:27.080978 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] RBAC: Adding query filter params (role.id=3bbb70ac844e475ea07d282e37dd3243, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6f6d6866832f456d84536d3cc45e512c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:27.081407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:27.082966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c98ebc-6986-42f2-9590-f967bbc6a7b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:27.104075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:42:26 2022] GET /identity/v3/role_assignments?user.id=6f6d6866832f456d84536d3cc45e512c&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=3bbb70ac844e475ea07d282e37dd3243 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:29.558762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4d69-9cd3-4a6a-884c-72878e77b881 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:29.559307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4d69-9cd3-4a6a-884c-72878e77b881 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:29.559307 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4d69-9cd3-4a6a-884c-72878e77b881 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:29.560730 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 631/1247] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:29.565992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:29.566448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:29.566605 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:29.572254 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:29.586392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:29.587613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:29.664861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:29.689885 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b4d69-9cd3-4a6a-884c-72878e77b881 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:29.710755 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:29.808527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b4d69-9cd3-4a6a-884c-72878e77b881 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:29.845729 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:29.846502 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:29.846783 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:29.847056 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:29.859428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:29.868285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:29.869525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:29.870576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 632/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:29.878080 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:29.989196 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66607d25-d41a-43a3-8692-12a1d85dfc69 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:30.022082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:30.028377 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:30.029130 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:30.029420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:30.029743 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:30.030547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:30.030860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:30.032268 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:30.044137 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:42:30.049904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:30.149340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:30.179102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dfa6-340c-4e3e-a251-ee1305d122f0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:30.220801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:30.221672 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:30.221956 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:30.222234 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:30.234420 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:30.236208 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:30.257010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:30.258119 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:30.259120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:30.259120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:30.259120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:30.259120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:30.259120 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 633/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:30.264060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:30.382889 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1c3-e029-4884-86d4-e15fa2b78c4e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:30.421534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:30.422326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:30.422659 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:30.422971 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:30.423732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:30.424084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:30.425703 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:30.440783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:30.447219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:30.560620 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd996f-02d9-4de2-bf30-0420649b7de3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:30.598972 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:30.612649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:30.614642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:30.615849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:30.616417 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:30.632557 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:30.635444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:30.655301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:30.656531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:30.657564 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 634/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:30.664131 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:30.727104 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:30.782153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82684d19-1d48-431f-9811-37e52a4a8237 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:30.821850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:30.822609 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:30.822934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:30.823246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:30.824118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:30.824439 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:30.825943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:30.850894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:30.852442 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:30.966322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccafcc-6bfe-4358-99b5-9ac6f231ce33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:31.007009 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:31.007658 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:31.007967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:31.008343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:31.068074 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:31.082889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:31.084349 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:31.212046 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 635/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:42:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:42:31.218082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:31.325274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62c8c1-6eae-4d77-8151-ac7e0f5a2385 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:31.340153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:31.364656 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:31.365552 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:31.365795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:31.366075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:31.377683 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:31.379300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:31.401498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:31.402669 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:31.403569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:31.403569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:31.403569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:31.403569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:31.403569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:31.408590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:31.501518 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:31.514921 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2d434-6e7e-4e6f-a074-9804527d73f7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:31.552180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:31.553401 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:31.553685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:31.553962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:31.554705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:31.555019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:31.556452 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:31.569773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 636/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:31.575480 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:31.681915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5de71-c521-4fa6-9cb8-ee40c0aae3b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:31.717738 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:31.718532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:31.718821 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:31.719096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:31.726118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:31.727604 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:31.742416 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:31.743359 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:31.744456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:31.749554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:31.855071 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dad80-197f-4f36-819f-5592f7739555 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:31.891749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:31.892440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:31.892728 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:31.893007 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:31.893799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:31.894139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:31.895534 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ab84d-7d88-4b43-b400-32e7481ef6c9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:31.907272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 637/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:31.913990 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:31.939801 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:32.012727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c2ecc-31a0-4b02-ad32-d7c9a62dd48e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:32.045203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:32.045840 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:32.046126 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:32.046405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:32.047226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6f6d6866832f456d84536d3cc45e512c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:32.047542 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:32.048999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e69b3-4f8d-4a4b-bcdc-9309b1ae160b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:32.054881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:42:31 2022] GET /identity/v3/role_assignments?user.id=6f6d6866832f456d84536d3cc45e512c&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:34.251381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718d3a-4a95-48cc-93de-eb8ccc05b721 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.251912 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718d3a-4a95-48cc-93de-eb8ccc05b721 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.251912 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718d3a-4a95-48cc-93de-eb8ccc05b721 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.253333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 638/1261] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:34.258595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.258969 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.259152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.276089 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:34.277283 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:34.342755 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:34.365993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718d3a-4a95-48cc-93de-eb8ccc05b721 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:34.376893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:34.465780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718d3a-4a95-48cc-93de-eb8ccc05b721 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:34.500110 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:34.500883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.501167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.501489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.511310 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:34.518906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:34.520017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:34.520953 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 639/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:34.527067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:34.545614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:34.623865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70452b-60d7-42ce-9163-1ba837554fd4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:34.656776 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:34.657431 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.657713 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.657987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.658727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:34.659036 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:34.660443 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:34.669072 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:34.674803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:34.740781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:34.774597 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae077-2206-4947-bf2d-f0ea807527f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:34.807905 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:34.808706 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.808994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.809275 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.819666 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:34.821141 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:34.838470 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:34.839513 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:34.840797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:34.840797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:34.840797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:34.840797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:34.840797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 640/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:34.845315 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:34.924166 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:34.942552 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc71430-8522-4671-a583-1daa6ec5f575 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:34.975835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:34.976504 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:34.976790 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:34.977069 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:34.977786 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:34.978097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:34.979742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:34.991215 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:34.996854 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.095982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712c5a-d35e-4db5-b9e9-864a5fa0a4f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.117795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:35.136966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.137599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.138053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.138386 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.149479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.151328 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:35.169781 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:35.170913 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:35.172167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 641/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:35.177410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.278193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463c9da-31de-4c10-8b08-2cdf95b34490 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.311542 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.312226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.312506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.312776 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.313596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:35.313905 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.315325 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.324145 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:35.332335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:35.338421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.441138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3406-9642-46fc-9519-d975c839d385 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.475490 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.476374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.476693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.476995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.487814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.489414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:35.508488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:35.509545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:35.510494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:35.510494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:35.510494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:35.510494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:35.510494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 642/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:35.515374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.611403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c86a4-0f94-48da-958d-fdc0db4f23ba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.645020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.645671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.645960 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.646238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.647059 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:35.647369 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.648766 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.659861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:35.666546 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.699246 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:35.770907 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e034-582b-403f-8afa-7732dd2f2c2f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.805213 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.806048 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.806365 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.806661 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.813685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.815207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:35.830608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:35.831630 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:35.832574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 643/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:35.838441 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:35.895401 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:35.936819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611886e-42a7-426a-8e3a-81292fb011c0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:35.969742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:35.970391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:35.970680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:35.970958 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:35.971739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:35.972087 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:35.973523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a02c1-f78b-49f7-a490-2816bdce93d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:35.986074 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:35.992459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:36.085990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:36.095194 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af69c-dffc-4101-9408-ae90dd175395 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:36.137327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:36.138314 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:36.138637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:36.139017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] PATH_INFO: `/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6f6d6866832f456d84536d3cc45e512c/roles/4cc3812026fb45fc9a368a583529ea4d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:36.141093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5db404946b492d8d06b43ef9e251d3, user_id=6f6d6866832f456d84536d3cc45e512c, role_id=4cc3812026fb45fc9a368a583529ea4d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:36.142623 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874dfa5-1178-489b-82a6-1db81c16f777 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:36.156139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 644/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Wed May 25 13:42:35 2022] PUT /identity/v3/projects/0c5db404946b492d8d06b43ef9e251d3/users/6f6d6866832f456d84536d3cc45e512c/roles/4cc3812026fb45fc9a368a583529ea4d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:42:38.326457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906d08-3990-46b5-996b-e3cf49ea8527 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:38.327024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906d08-3990-46b5-996b-e3cf49ea8527 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:38.327024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9906d08-3990-46b5-996b-e3cf49ea8527 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:38.328447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1274] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:38.333689 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:38.334105 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:38.334385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:38.351156 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:38.352415 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:38.418398 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 645/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:38.444507 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9906d08-3990-46b5-996b-e3cf49ea8527 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:38.545468 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9906d08-3990-46b5-996b-e3cf49ea8527 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:38.562729 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:38.579724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:38.580477 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:38.580759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:38.581031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:38.591176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:38.598805 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:38.599850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:38.600939 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Wed May 25 13:42:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:38.606659 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:38.687420 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:38.705895 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe301e2-dbfe-4564-aa0b-d9654429d075 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:38.739635 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:38.741282 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:38.741566 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:38.741837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:38.742572 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:38.742874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:38.744237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:38.752176 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 646/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Wed May 25 13:42:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:38.757754 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:38.855212 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89541d11-88ad-47fc-8d64-35bc8753b5e5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:38.887780 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:38.888583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:38.888869 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:38.889147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:38.900098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:38.901538 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:38.918663 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:38.919718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:38.920851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:38.920851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:38.920851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:38.920851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:38.920851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:38.925598 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.024100 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae841-666f-4307-8532-6d0b37daeb9f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:39.056300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:39.056943 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:39.057226 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:39.057503 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:39.058219 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:39.058526 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:39.059866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:39.072004 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 647/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:39.077395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.109270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:39.192925 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820faaa-549b-42d9-9aa2-98332cb018cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:39.226429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:39.227213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:39.227521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:39.227791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:39.235182 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:39.236645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:39.251613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:39.252933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:39.254331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:39.259637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.263485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:39.358828 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2ab81-6eef-432f-8907-72b70ec45b43 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:39.393218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:39.393807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:39.394087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:39.394358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:39.395178 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:39.395485 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:39.397467 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:39.414615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 648/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Wed May 25 13:42:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:39.420293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.508392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:39.517646 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda53c1-8b89-4785-9be4-a99a1d772825 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:39.550279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:39.550911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:39.551199 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:39.551476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:39.616206 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:39.617660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:39.702374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:39.712648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Wed May 25 13:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:42:39.718440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.779560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:39.817837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58d7e2-3193-4fa6-b254-124bb9523c69 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:39.852305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:39.853082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:39.853364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:39.853637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:39.863718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:39.865256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:39.882385 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:39.883423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:39.884700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:39.884700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:39.884700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:39.884700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:39.884700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 649/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:39.889142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:39.985473 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d5fd-30e0-4a73-a05a-e098cfab4efe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:40.017229 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:40.017871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:40.018152 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:40.018426 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:40.019133 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:40.019439 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:40.020829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:40.032362 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Wed May 25 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:40.038335 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:40.135289 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5b352-c768-4133-a5bc-3b562c6040fe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:40.149477 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:40.169800 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:40.170579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:40.170860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:40.171135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:40.177823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:40.179261 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:40.192610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:40.194196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 25 13:42:40.195303 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 650/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Wed May 25 13:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:40.200084 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:40.249223 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:40.300165 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d955-d22e-4a6a-bd06-11e131168c5e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:40.332528 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:40.333177 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:40.333459 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:40.333731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:40.335083 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:40.335392 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:40.336850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de530f-f7cf-422c-97c8-c68080100c42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:40.348506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Wed May 25 13:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:40.355035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:40.453079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d539d7b-155e-404c-8a3d-5d279e03c722 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:40.487139 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:40.487774 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:40.488090 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:40.488369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:40.489196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] RBAC: Adding query filter params (role.id=4cc3812026fb45fc9a368a583529ea4d, scope.project.id=0c5db404946b492d8d06b43ef9e251d3, user.id=6f6d6866832f456d84536d3cc45e512c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:40.489509 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:40.490947 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fc00d-b3db-4216-a744-b322b7db5fb3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:40.496731 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 651/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Wed May 25 13:42:40 2022] GET /identity/v3/role_assignments?user.id=6f6d6866832f456d84536d3cc45e512c&scope.project.id=0c5db404946b492d8d06b43ef9e251d3&role.id=4cc3812026fb45fc9a368a583529ea4d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:42.809826 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d591-c9de-47d8-8483-5ee2294691fa None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:42.810368 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d591-c9de-47d8-8483-5ee2294691fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:42.810483 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d591-c9de-47d8-8483-5ee2294691fa None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:42.811901 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1288] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:42.817069 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:42.817463 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:42.817617 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:42.834493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:42.835653 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:42.864429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:42.901879 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 652/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:42.925456 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0d591-c9de-47d8-8483-5ee2294691fa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:43.025972 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0d591-c9de-47d8-8483-5ee2294691fa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:43.036324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:43.060337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:43.061064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:43.061351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:43.061628 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:43.062280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:43.063657 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:43.076159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:43.077346 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:43.078190 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:42:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:43.083489 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:43.199498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9f31c-dc36-468e-a402-3711f9801fd2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:43.233642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:43.234292 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:43.234574 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:43.234847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:43.235544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:43.235847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:43.237227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5de8-77b6-4e36-89b6-dbb17c8f1ef2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:43.243045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 653/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Wed May 25 13:42:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:43.248784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:43.346987 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945213d-6223-43ef-9d67-25e9b7265dc1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:43.381678 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:43.382400 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:43.382689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:43.382968 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:43.383675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:43.384024 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:43.385407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368704f1-9474-4767-b9ca-010866aa7b5a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:43.389779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Wed May 25 13:42:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:45.552423 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9727-7333-412e-8aeb-a6568ba3125a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:45.552950 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9727-7333-412e-8aeb-a6568ba3125a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:45.553046 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9727-7333-412e-8aeb-a6568ba3125a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:45.554380 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 654/1293] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:45.559487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f852e-baca-4818-912e-ec2273051f85 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:45.559900 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f852e-baca-4818-912e-ec2273051f85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:45.560090 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f852e-baca-4818-912e-ec2273051f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:45.577110 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-071f852e-baca-4818-912e-ec2273051f85 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:45.578293 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-071f852e-baca-4818-912e-ec2273051f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:45.607645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:45.645274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:45.668654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc9727-7333-412e-8aeb-a6568ba3125a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:45.698928 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:45.768140 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc9727-7333-412e-8aeb-a6568ba3125a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:45.801431 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:45.802074 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:45.802356 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:45.803123 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:45.804384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:45.805719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3fb3d-14c3-4784-b6e0-7501ae11e416 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:45.820355 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 655/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Wed May 25 13:42:45 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:42:48.041493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5843b7-af36-4fbc-b033-ba38142a3f2d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.042025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5843b7-af36-4fbc-b033-ba38142a3f2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.042146 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5843b7-af36-4fbc-b033-ba38142a3f2d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.043430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1296] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:48.048655 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.049020 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.049220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.066189 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:48.067353 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:48.139369 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 656/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:48.161446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5843b7-af36-4fbc-b033-ba38142a3f2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:48.261043 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5843b7-af36-4fbc-b033-ba38142a3f2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:48.293451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:48.294261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.294550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.294825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.295476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:48.296879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:48.317462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:48.319288 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:48.320376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:42:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:48.326019 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:48.344177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:48.424999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e5c98-c881-48b7-be89-2eacf1784910 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:48.458524 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:48.459751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.460062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.460336 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.461070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:48.461376 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:48.462733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:48.467762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 657/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Wed May 25 13:42:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:48.473756 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:48.506993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:48.573435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aa054-546e-4ed7-a307-ab66dfc552b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:48.606333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:48.606980 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.607268 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.607548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.608370 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] RBAC: Adding query filter params (interface=public, service_id=8495ec9f96cc4730aec0fec46b94faa4, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:48.608689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:48.610064 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940c94-3182-4fac-8d2c-e72f86ee7cad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:48.614984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Wed May 25 13:42:48 2022] GET /identity/v3/endpoints?service_id=8495ec9f96cc4730aec0fec46b94faa4&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:48.620674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:48.701479 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:48.720081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd9fed-9916-4404-99f1-ebe1c8c04ba3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:48.753184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:48.753826 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:48.754106 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:48.754375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:48.755043 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:48.757451 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7e68a-890b-4afc-9946-d94a030fedf4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:48.764092 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 658/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Wed May 25 13:42:48 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:42:50.950334 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b2af8-6a9f-435f-82cb-90a260bb651a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:50.950866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b2af8-6a9f-435f-82cb-90a260bb651a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:50.950866 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b2af8-6a9f-435f-82cb-90a260bb651a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:50.952276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1302] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:50.957525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:50.957925 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:50.958088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:50.975433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:50.976690 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:51.041815 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 659/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:42:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:51.064313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b2af8-6a9f-435f-82cb-90a260bb651a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:51.164487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:51.171235 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b2af8-6a9f-435f-82cb-90a260bb651a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:51.203422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:51.204176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:51.204463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:51.204736 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:51.205380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:51.206769 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:51.219302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:51.220489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application May 25 13:42:51.221463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Wed May 25 13:42:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:51.227723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:51.302016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:51.324678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014602cf-a837-44e9-9474-c3d55acad074 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:51.359102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:51.359768 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:51.360093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:51.360372 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:51.361081 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:51.361390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:51.362773 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fe404-76df-4c45-871a-2e91dbf8b67d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:51.368093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 660/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Wed May 25 13:42:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:51.373679 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:51.471413 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f897-bc15-4959-80cb-bdd7b5fc2502 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:51.503868 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:51.504539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:51.504827 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:51.505107 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:51.506382 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:51.507781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf368-99bd-4d6a-b741-12498f323821 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:51.526757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Wed May 25 13:42:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:43:01.368041 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=30 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:01.522240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:55.183201 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d358-dd3e-4dbc-bd38-69ca64e7ed3f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:55.183762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d358-dd3e-4dbc-bd38-69ca64e7ed3f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:55.183762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d358-dd3e-4dbc-bd38-69ca64e7ed3f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:55.185388 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 661/1307] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:55.190759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1ce72-2d7c-4282-89f3-71656febf5eb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:55.193039 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1ce72-2d7c-4282-89f3-71656febf5eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:55.193691 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1ce72-2d7c-4282-89f3-71656febf5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:55.291096 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-03b1ce72-2d7c-4282-89f3-71656febf5eb None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:55.292613 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03b1ce72-2d7c-4282-89f3-71656febf5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:55.415246 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:55.467462 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:55.467768 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:55.468080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:55.470978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 662/1309] 10.222.0.45 () {54 vars in 964 bytes} [Wed May 25 13:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:55.478275 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2e7a-45df-4acb-b34c-891c61a1efd4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:55.478570 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2e7a-45df-4acb-b34c-891c61a1efd4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:55.478929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2e7a-45df-4acb-b34c-891c61a1efd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:55.580574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-663a2e7a-45df-4acb-b34c-891c61a1efd4 None None] MFA Rules not processed for user `eb026adfce894a72b97d750cdbb72efc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:55.581919 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-663a2e7a-45df-4acb-b34c-891c61a1efd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:55.608295 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:55.677200 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Wed May 25 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:43:55.686749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:55.688740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:55.832779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287e82d-74aa-477d-aee8-f84050280703 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:55.921345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:55.922284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:55.922647 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:55.922994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:55.924828 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:55.990885 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:56.062949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:56.064807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32407243-bfd6-4403-93a7-9cceeebe560f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:56.091565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 663/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:58.680907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e78e4-2ebc-45fb-be1f-c2d1574fd3f1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:58.681463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e78e4-2ebc-45fb-be1f-c2d1574fd3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:58.681463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e78e4-2ebc-45fb-be1f-c2d1574fd3f1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:58.682759 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1312] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:58.688444 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d873-8ef3-4588-ac1c-834793da0f1e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:58.688926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d873-8ef3-4588-ac1c-834793da0f1e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:58.689265 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d873-8ef3-4588-ac1c-834793da0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:58.778739 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c3b7d873-8ef3-4588-ac1c-834793da0f1e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:58.780130 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3b7d873-8ef3-4588-ac1c-834793da0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:58.884806 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 664/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:58.939263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:58.939481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:58.939761 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:58.941204 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1314] 10.222.0.45 () {54 vars in 964 bytes} [Wed May 25 13:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:58.947437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b75-7622-4516-aa7b-dead317ba0de None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:58.947799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b75-7622-4516-aa7b-dead317ba0de None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:58.948164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9b75-7622-4516-aa7b-dead317ba0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:58.958778 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:59.039361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-942b9b75-7622-4516-aa7b-dead317ba0de None None] MFA Rules not processed for user `eb026adfce894a72b97d750cdbb72efc`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:59.040626 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-942b9b75-7622-4516-aa7b-dead317ba0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:59.123196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 665/1315] 10.222.0.45 () {58 vars in 1061 bytes} [Wed May 25 13:43:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:59.132372 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:59.133933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:59.245642 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fc00a-9554-4a57-a331-4b101eb26057 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:59.328665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:59.331648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:59.332532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:59.332933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:59.333404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:59.336434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:59.460767 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:59.462220 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41f8c8-46af-479c-9b7a-603cebc92094 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:59.481951 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1316] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:02.262523 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:02.263079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:02.263079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:02.264500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 666/1317] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:02.269884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbb035-388c-40ae-83a7-98a2409e4f53 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:02.270292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbb035-388c-40ae-83a7-98a2409e4f53 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:02.270437 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbb035-388c-40ae-83a7-98a2409e4f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:02.336136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9adbb035-388c-40ae-83a7-98a2409e4f53 None None] MFA Rules not processed for user `11af095e02cd4d71b25cc4525e9fdf03`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:02.337339 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9adbb035-388c-40ae-83a7-98a2409e4f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:02.390100 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:02.437245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1318] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:02.477392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:02.478942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:02.581575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40301c7f-9cea-4708-90f4-4a4da17cd24b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:02.584351 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:02.656446 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:02.657091 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:02.657373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:02.657650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:02.659301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:02.766611 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:02.768062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17bc28-fc85-43cd-9f49-fcb7a47721e8 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:02.786537 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 667/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:05.686309 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:05.686896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:05.686896 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:05.688324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1320] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:05.693497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d323-5dd8-44f4-955b-e3a017b84045 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:05.693899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d323-5dd8-44f4-955b-e3a017b84045 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:05.694063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d323-5dd8-44f4-955b-e3a017b84045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:05.734437 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:05.760285 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aee7d323-5dd8-44f4-955b-e3a017b84045 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:05.761480 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aee7d323-5dd8-44f4-955b-e3a017b84045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:05.854836 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 668/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:05.892550 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:05.894154 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:05.996002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c21e0-36e7-4318-a97e-404e3ea4ec70 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:06.068495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:06.069138 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:06.070060 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:06.070341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:06.072028 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:06.079365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:06.188557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:06.190017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f4eb2-d4cf-4f43-a989-1c2ec8de5756 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:06.208336 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:09.118762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:09.119300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:09.119300 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:09.120856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 669/1323] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:09.125831 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df543-e551-4084-a1fd-f44fe5cb1c3d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:09.126249 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df543-e551-4084-a1fd-f44fe5cb1c3d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:09.126403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df543-e551-4084-a1fd-f44fe5cb1c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:09.192525 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-259df543-e551-4084-a1fd-f44fe5cb1c3d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:09.193718 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-259df543-e551-4084-a1fd-f44fe5cb1c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:09.251306 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:09.291864 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:09.328373 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:09.330042 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:09.438206 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e735-8a3a-43e5-96c8-be385c609006 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:09.450406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:09.513338 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:09.513982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:09.514263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:09.514531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:09.516227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:09.627426 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:09.628924 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2842e6a-6711-48d5-9600-8252867c1a89 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:09.648469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 670/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:11.927009 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:11.927535 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:11.927750 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:11.929829 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1326] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:11.935107 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817001-36c8-4ebe-935e-8d1d9440a528 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:11.935510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817001-36c8-4ebe-935e-8d1d9440a528 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:11.935663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817001-36c8-4ebe-935e-8d1d9440a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:11.982452 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:12.001641 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d5817001-36c8-4ebe-935e-8d1d9440a528 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:12.002840 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d5817001-36c8-4ebe-935e-8d1d9440a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:12.099853 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 671/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:12.135964 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:12.137527 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:12.241902 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcfb02-18c4-405c-80e1-821570d052c4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:12.312639 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:12.313280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:12.313576 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:12.313841 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:12.315481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:12.331850 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:12.429391 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:12.430843 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b042a87-b1ea-4ede-81df-ad3b340c08cb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:12.449547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:14.732552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:14.733112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:14.733112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:14.734506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 672/1329] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:14.739784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3df01-f60f-48f4-9dc1-88218995b494 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:14.740222 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3df01-f60f-48f4-9dc1-88218995b494 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:14.740491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3df01-f60f-48f4-9dc1-88218995b494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:14.806082 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4da3df01-f60f-48f4-9dc1-88218995b494 None None] MFA Rules not processed for user `11af095e02cd4d71b25cc4525e9fdf03`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:14.807276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4da3df01-f60f-48f4-9dc1-88218995b494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:14.873439 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:14.906594 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:14.944255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:14.946456 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:15.052101 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3a35-354d-4d70-aebc-5d49f0967042 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:15.073001 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:15.126686 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:15.127343 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:15.127628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:15.127906 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:15.129612 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:15.238578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:15.240039 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ce0d4-5a12-4fd2-adac-1fecff499f13 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:15.259601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 673/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:17.755298 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:17.755877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:17.755877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:17.757429 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1332] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:17.762871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb30-e854-42cf-9c30-df1f06b676fe None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:17.763301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb30-e854-42cf-9c30-df1f06b676fe None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:17.763411 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb30-e854-42cf-9c30-df1f06b676fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:17.826560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:17.829240 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-455bbb30-e854-42cf-9c30-df1f06b676fe None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:17.830968 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-455bbb30-e854-42cf-9c30-df1f06b676fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:17.930833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 674/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:17.966261 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:17.967811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:18.073224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bf7d5-0c9d-416e-a82b-d4d79b7ce251 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:18.147504 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:18.149255 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:18.149518 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:18.149791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:18.151488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:18.182386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:18.270221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:18.271697 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1f532-e927-4ed7-b04e-4f10057fbabb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:18.291677 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:20.542528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:20.543053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:20.543167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:20.544496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 675/1335] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:20.549726 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a86d3-5594-404e-b926-63985d7b9486 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:20.550106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a86d3-5594-404e-b926-63985d7b9486 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:20.550335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a86d3-5594-404e-b926-63985d7b9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:20.615244 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-706a86d3-5594-404e-b926-63985d7b9486 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:20.617059 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-706a86d3-5594-404e-b926-63985d7b9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:20.691386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:20.714207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:20.754776 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:20.756345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:20.861497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f991052-ac1c-4927-96bf-82eabb41fcb5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:20.891654 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:20.936340 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:20.936988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:20.937274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:20.937550 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:20.939207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:21.053735 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:21.055816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dfdc1-3a20-48ae-8e76-1ef88eeff02e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:21.074458 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 676/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:23.770881 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:23.771415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:23.771415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:23.773493 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1338] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:23.778747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf056-45ae-48b2-91ec-c238d83c4638 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:23.779191 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf056-45ae-48b2-91ec-c238d83c4638 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:23.779295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adf056-45ae-48b2-91ec-c238d83c4638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:23.844097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16adf056-45ae-48b2-91ec-c238d83c4638 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:23.845304 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16adf056-45ae-48b2-91ec-c238d83c4638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:23.853833 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:23.942016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 677/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:23.978579 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:23.980171 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:24.084038 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99755765-7fa6-4df5-9ad2-1130d0af3526 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:24.157179 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:24.157822 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:24.158103 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:24.158381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:24.160048 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:24.195775 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:24.268912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:24.270455 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aca7ab-0f0d-4782-8e22-d07faa36af53 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:24.288539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:28.556718 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.557290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.557290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.558771 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 678/1341] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:28.564198 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9eb3-8abe-4a0c-9409-306c96550d93 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.564652 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9eb3-8abe-4a0c-9409-306c96550d93 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.564758 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9eb3-8abe-4a0c-9409-306c96550d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.636231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a2cb9eb3-8abe-4a0c-9409-306c96550d93 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:28.637421 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a2cb9eb3-8abe-4a0c-9409-306c96550d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:28.727213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:28.740549 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:28.779868 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:28.782058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:28.880440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dca89-667c-44ee-9440-8052623dfea6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:28.884959 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:28.885607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.885889 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.886169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.895205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:28.896674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3c676-256b-44a0-a752-52c89a8b09aa service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:28.906237 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 679/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:31.710999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229b82-5038-4830-af75-29844d98d01c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.711529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229b82-5038-4830-af75-29844d98d01c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.711640 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229b82-5038-4830-af75-29844d98d01c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.712996 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1344] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:31.718265 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed0c4-92ae-4be9-9073-0254e1d1f113 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.718671 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed0c4-92ae-4be9-9073-0254e1d1f113 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.718827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed0c4-92ae-4be9-9073-0254e1d1f113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.747035 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-51bed0c4-92ae-4be9-9073-0254e1d1f113 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:31.748241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-51bed0c4-92ae-4be9-9073-0254e1d1f113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:31.775851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:31.825860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 680/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:31.863997 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229b82-5038-4830-af75-29844d98d01c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:31.875304 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229b82-5038-4830-af75-29844d98d01c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:31.878776 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:31.879410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.879693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.879995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.888884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:31.890329 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd4e8d-7763-47d4-b3cc-0b97ce945eaa service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:31.900380 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:36.306076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194dadc-5284-4798-929d-f884c0caaae7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:36.306652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194dadc-5284-4798-929d-f884c0caaae7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:36.306652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194dadc-5284-4798-929d-f884c0caaae7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:36.308030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 681/1347] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:36.313598 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e8a2-c066-4fac-b1f2-77737c09fa8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:36.314012 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e8a2-c066-4fac-b1f2-77737c09fa8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:36.314290 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e8a2-c066-4fac-b1f2-77737c09fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:36.334128 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c65e8a2-c066-4fac-b1f2-77737c09fa8a None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:36.335332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c65e8a2-c066-4fac-b1f2-77737c09fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:36.343722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:36.381775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194dadc-5284-4798-929d-f884c0caaae7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:36.393179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194dadc-5284-4798-929d-f884c0caaae7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:36.396645 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:36.397283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:36.397568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:36.397845 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:36.407377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:36.408796 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519f310-c518-4385-9dfd-70da6c29e4c8 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:36.418812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 682/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:41.808063 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665059-885f-4d6f-b2ae-ba6e11e36d79 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:41.808627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665059-885f-4d6f-b2ae-ba6e11e36d79 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:41.809097 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665059-885f-4d6f-b2ae-ba6e11e36d79 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:41.810863 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1350] 10.222.0.45 () {54 vars in 921 bytes} [Wed May 25 13:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:41.816175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246644-c1f2-49b4-8467-cea9082de554 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:41.816582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246644-c1f2-49b4-8467-cea9082de554 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:41.816737 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246644-c1f2-49b4-8467-cea9082de554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:41.906864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-76246644-c1f2-49b4-8467-cea9082de554 None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:41.908324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76246644-c1f2-49b4-8467-cea9082de554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:42.016136 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 683/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Wed May 25 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:46.127855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6fa0-8ba2-4271-bd0c-11f8d5545dee None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:46.128407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6fa0-8ba2-4271-bd0c-11f8d5545dee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:46.128523 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6fa0-8ba2-4271-bd0c-11f8d5545dee None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:46.129993 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1352] 10.222.0.45 () {54 vars in 921 bytes} [Wed May 25 13:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:46.135693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e90a-8dbe-4405-a5d9-f704fa5eb21c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:46.136184 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e90a-8dbe-4405-a5d9-f704fa5eb21c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:46.136486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e90a-8dbe-4405-a5d9-f704fa5eb21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:46.154575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-86c5e90a-8dbe-4405-a5d9-f704fa5eb21c None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:46.155837 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-86c5e90a-8dbe-4405-a5d9-f704fa5eb21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:46.165384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 684/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Wed May 25 13:44:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:46.328434 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=13 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:47.829349 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c01bb3-0401-45e8-87fd-00c9becb606e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:47.830001 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c01bb3-0401-45e8-87fd-00c9becb606e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:47.830156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c01bb3-0401-45e8-87fd-00c9becb606e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:47.831548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1354] 10.222.0.45 () {54 vars in 921 bytes} [Wed May 25 13:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:47.837627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3797-8557-4226-91d7-012c7fcc0700 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:47.838097 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3797-8557-4226-91d7-012c7fcc0700 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:47.838238 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc3797-8557-4226-91d7-012c7fcc0700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:47.856511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-49fc3797-8557-4226-91d7-012c7fcc0700 None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:47.857705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49fc3797-8557-4226-91d7-012c7fcc0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:47.866571 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 685/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Wed May 25 13:44:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:51.859458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7120-c0c1-40fb-9c77-2133ae52c5f9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:51.860347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7120-c0c1-40fb-9c77-2133ae52c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:51.860627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7120-c0c1-40fb-9c77-2133ae52c5f9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:51.862004 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1356] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:51.867971 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169f79-36cf-4d1a-83ed-599195d9dda1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:51.868846 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169f79-36cf-4d1a-83ed-599195d9dda1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:51.869334 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169f79-36cf-4d1a-83ed-599195d9dda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:51.890149 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e6169f79-36cf-4d1a-83ed-599195d9dda1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:51.891568 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e6169f79-36cf-4d1a-83ed-599195d9dda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:51.900794 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 686/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:51.958689 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739332ef-e2aa-47e2-9f99-a1946a11d39d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:51.959052 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739332ef-e2aa-47e2-9f99-a1946a11d39d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:51.959597 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739332ef-e2aa-47e2-9f99-a1946a11d39d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:51.961934 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1358] 10.222.0.45 () {54 vars in 961 bytes} [Wed May 25 13:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:51.977939 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544282c8-37e2-4856-a525-c3ab049bb9d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:51.978232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544282c8-37e2-4856-a525-c3ab049bb9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:51.978848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544282c8-37e2-4856-a525-c3ab049bb9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:52.025592 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:52.038936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-544282c8-37e2-4856-a525-c3ab049bb9d9 None None] MFA Rules not processed for user `6086ccae239045e5b109e33cf47453a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:52.040268 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-544282c8-37e2-4856-a525-c3ab049bb9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:52.132636 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 687/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Wed May 25 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:52.140922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739332ef-e2aa-47e2-9f99-a1946a11d39d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:52.156163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739332ef-e2aa-47e2-9f99-a1946a11d39d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:52.161577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:52.162300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:52.162648 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:52.162989 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:52.172520 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:52.174195 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31a14f-8fdd-40b3-818d-f3eeba4b0a5f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:52.184057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:54.657578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2f61-37a8-4920-8d2f-1cac1035a883 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:54.658133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2f61-37a8-4920-8d2f-1cac1035a883 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:54.658133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af2f61-37a8-4920-8d2f-1cac1035a883 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:54.659528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 688/1361] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:54.665430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cccc-1156-4cf4-89a5-080b807bb3a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:54.665871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cccc-1156-4cf4-89a5-080b807bb3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:54.665992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cccc-1156-4cf4-89a5-080b807bb3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:54.685331 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0d43cccc-1156-4cf4-89a5-080b807bb3a2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:54.686532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d43cccc-1156-4cf4-89a5-080b807bb3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:54.694476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:54.743693 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2f61-37a8-4920-8d2f-1cac1035a883 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:54.763256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af2f61-37a8-4920-8d2f-1cac1035a883 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:54.768999 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:54.769769 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:54.770062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:54.770530 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:54.780151 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:54.781920 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111497a-3dc0-4a0f-8883-cd2105ce7f2d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:54.793013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 689/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:57.390984 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6388-36ea-4bfd-be00-d8495a56db43 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:57.391899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6388-36ea-4bfd-be00-d8495a56db43 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:57.392410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6388-36ea-4bfd-be00-d8495a56db43 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:57.394289 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1364] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:57.400497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35575b-501b-4d79-a56c-456af22374c0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:57.400936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35575b-501b-4d79-a56c-456af22374c0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:57.401098 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35575b-501b-4d79-a56c-456af22374c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:57.419827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b35575b-501b-4d79-a56c-456af22374c0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:57.422059 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b35575b-501b-4d79-a56c-456af22374c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:57.434910 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 690/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:57.485424 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97986de-c197-449a-afe7-395bbf2bb677 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:57.485619 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97986de-c197-449a-afe7-395bbf2bb677 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:57.485899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97986de-c197-449a-afe7-395bbf2bb677 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:57.487310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1366] 10.222.0.45 () {54 vars in 961 bytes} [Wed May 25 13:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:57.493880 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500437-c2d1-4155-8b66-e764d298540c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:57.494172 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500437-c2d1-4155-8b66-e764d298540c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:57.494459 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500437-c2d1-4155-8b66-e764d298540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:57.514414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ab500437-c2d1-4155-8b66-e764d298540c None None] MFA Rules not processed for user `6086ccae239045e5b109e33cf47453a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:57.515651 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab500437-c2d1-4155-8b66-e764d298540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:57.526281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 691/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Wed May 25 13:44:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:57.534071 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97986de-c197-449a-afe7-395bbf2bb677 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:57.548620 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97986de-c197-449a-afe7-395bbf2bb677 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:57.554684 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:57.555319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:57.555602 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:57.555904 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:57.563983 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:57.565626 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea8242-4128-4d1f-ac26-0dd42bb2f308 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:57.574999 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:59.854642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231c10-fde6-4258-8be1-e087ec917987 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:59.855228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231c10-fde6-4258-8be1-e087ec917987 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:59.855228 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231c10-fde6-4258-8be1-e087ec917987 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:59.856904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 692/1369] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:59.862414 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16110-af90-403c-9f26-9b57da88a8b8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:59.862856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16110-af90-403c-9f26-9b57da88a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:59.862950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d16110-af90-403c-9f26-9b57da88a8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:59.883211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e6d16110-af90-403c-9f26-9b57da88a8b8 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:59.884435 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6d16110-af90-403c-9f26-9b57da88a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:59.893170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:59.937161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e231c10-fde6-4258-8be1-e087ec917987 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:59.953203 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e231c10-fde6-4258-8be1-e087ec917987 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:59.959106 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:59.960725 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:59.961183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:59.961628 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:59.970692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:59.972506 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca01a5-48f6-4f7b-a564-ee3cef99a555 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:59.986348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 693/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:00.199347 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a966-f5c0-4b21-bb6f-13055c113ca0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:00.199937 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a966-f5c0-4b21-bb6f-13055c113ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:00.200214 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a966-f5c0-4b21-bb6f-13055c113ca0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:00.201936 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1372] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:00.208077 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc1f8f-a276-4dd5-aabf-feaefc1d533d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:00.208428 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc1f8f-a276-4dd5-aabf-feaefc1d533d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:00.208663 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc1f8f-a276-4dd5-aabf-feaefc1d533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:00.228874 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5afc1f8f-a276-4dd5-aabf-feaefc1d533d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:00.230150 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5afc1f8f-a276-4dd5-aabf-feaefc1d533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:00.238966 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 694/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:00.513118 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56467533-8823-4c1c-b62a-80220854fac0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:00.513861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56467533-8823-4c1c-b62a-80220854fac0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:00.514224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56467533-8823-4c1c-b62a-80220854fac0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:00.515724 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1374] 10.222.0.45 () {54 vars in 963 bytes} [Wed May 25 13:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:00.520797 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020fb3f-3964-4e16-b2ed-45ac135231e3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:00.521093 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020fb3f-3964-4e16-b2ed-45ac135231e3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:00.521374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020fb3f-3964-4e16-b2ed-45ac135231e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:00.573849 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1020fb3f-3964-4e16-b2ed-45ac135231e3 None None] MFA Rules not processed for user `3e02e0b0eac04b51970fa87a0df61e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:00.575578 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1020fb3f-3964-4e16-b2ed-45ac135231e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:00.642998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 695/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Wed May 25 13:45:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:00.651627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56467533-8823-4c1c-b62a-80220854fac0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:00.663158 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56467533-8823-4c1c-b62a-80220854fac0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:00.667453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e02e0b0eac04b51970fa87a0df61e64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:00.668397 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:00.668966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:00.669469 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:00.680070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:00.684768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202610-eb2b-4cbe-b0b2-24b143f9b98e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:00.700439 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:02.276576 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7ce7-2fbe-435e-8cfe-3025f10101ec None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:02.277133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7ce7-2fbe-435e-8cfe-3025f10101ec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:02.277133 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7ce7-2fbe-435e-8cfe-3025f10101ec None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:02.278501 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 696/1377] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:02.283882 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be6e5-20b7-4465-b6cf-c6e202283587 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:02.284341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be6e5-20b7-4465-b6cf-c6e202283587 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:02.284471 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34be6e5-20b7-4465-b6cf-c6e202283587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:02.302428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f34be6e5-20b7-4465-b6cf-c6e202283587 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:02.305040 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f34be6e5-20b7-4465-b6cf-c6e202283587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:02.314269 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:02.360701 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7ce7-2fbe-435e-8cfe-3025f10101ec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:02.376350 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7ce7-2fbe-435e-8cfe-3025f10101ec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:02.382801 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86841003-361c-431e-bf57-e523d734411f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:02.383469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841003-361c-431e-bf57-e523d734411f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:02.383752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841003-361c-431e-bf57-e523d734411f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:02.384064 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841003-361c-431e-bf57-e523d734411f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:02.393439 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86841003-361c-431e-bf57-e523d734411f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:02.395073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86841003-361c-431e-bf57-e523d734411f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:02.399560 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:02.407407 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 697/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:04.669279 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d516f5-f313-4cc7-887b-d2484dec9337 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:04.669816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d516f5-f313-4cc7-887b-d2484dec9337 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:04.669816 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d516f5-f313-4cc7-887b-d2484dec9337 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:04.671207 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1380] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:04.676551 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f5d5a-9cfb-4781-b83b-adbcde2dd705 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:04.676944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f5d5a-9cfb-4781-b83b-adbcde2dd705 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:04.677113 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f5d5a-9cfb-4781-b83b-adbcde2dd705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:04.697723 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f12f5d5a-9cfb-4781-b83b-adbcde2dd705 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:04.698904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f12f5d5a-9cfb-4781-b83b-adbcde2dd705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:04.708080 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 698/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:04.762454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d516f5-f313-4cc7-887b-d2484dec9337 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:04.779265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d516f5-f313-4cc7-887b-d2484dec9337 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:04.784950 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:04.785700 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:04.786085 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:04.786463 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:04.796488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:04.798178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3317e-3c7e-4c4a-b9ca-3e276752c2dd service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:04.808732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:07.103882 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b673-9a63-413b-8dce-bad7e4f5d450 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:07.104468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b673-9a63-413b-8dce-bad7e4f5d450 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:07.104468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b673-9a63-413b-8dce-bad7e4f5d450 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:07.105848 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 699/1383] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:07.111415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbb5f-057f-44fd-9a4b-02da59107434 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:07.111799 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbb5f-057f-44fd-9a4b-02da59107434 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:07.112002 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbb5f-057f-44fd-9a4b-02da59107434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:07.130044 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6fbfbb5f-057f-44fd-9a4b-02da59107434 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:07.131417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fbfbb5f-057f-44fd-9a4b-02da59107434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:07.139746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:07.181775 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b673-9a63-413b-8dce-bad7e4f5d450 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:07.197152 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b673-9a63-413b-8dce-bad7e4f5d450 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:07.202606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:07.203317 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:07.203681 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:07.204161 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:07.213673 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:07.215422 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d433-0287-4405-818a-bda36d577d60 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:07.226283 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 700/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:09.465418 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14fd68-629e-4c67-8717-2d936fdcee7c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:09.466170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14fd68-629e-4c67-8717-2d936fdcee7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:09.466170 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14fd68-629e-4c67-8717-2d936fdcee7c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:09.467447 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1386] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:09.475488 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705d57c-dd91-4164-82b9-ba0bc661f6f2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:09.475967 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705d57c-dd91-4164-82b9-ba0bc661f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:09.476363 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705d57c-dd91-4164-82b9-ba0bc661f6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:09.506482 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a705d57c-dd91-4164-82b9-ba0bc661f6f2 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:09.508223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a705d57c-dd91-4164-82b9-ba0bc661f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:09.518694 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 701/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:09.564537 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14fd68-629e-4c67-8717-2d936fdcee7c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:09.582232 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14fd68-629e-4c67-8717-2d936fdcee7c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:09.588779 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:09.589529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:09.589875 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:09.590211 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:09.601187 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:09.603094 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8798d-96f1-4e4a-b666-b7b5286f455f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:09.615422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:11.900669 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1893f-9af4-4a2f-9e13-fc256c9b4bf0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:11.901223 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1893f-9af4-4a2f-9e13-fc256c9b4bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:11.901656 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1893f-9af4-4a2f-9e13-fc256c9b4bf0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:11.904073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 702/1389] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:11.909231 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e2f9c-e10e-4355-a61a-35019e203ce0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:11.909915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e2f9c-e10e-4355-a61a-35019e203ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:11.909915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e2f9c-e10e-4355-a61a-35019e203ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:11.930768 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-630e2f9c-e10e-4355-a61a-35019e203ce0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:11.932912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-630e2f9c-e10e-4355-a61a-35019e203ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:11.945612 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:11.993558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1893f-9af4-4a2f-9e13-fc256c9b4bf0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:12.008679 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1893f-9af4-4a2f-9e13-fc256c9b4bf0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:12.014410 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:12.015063 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:12.015378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:12.015601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:12.025177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:12.026700 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c28882-7292-445a-bfbf-cae68d9c7b31 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:12.036816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 703/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:14.576599 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e10ea6-04c7-468e-a87b-f8c2a7c7bc0d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:14.577239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e10ea6-04c7-468e-a87b-f8c2a7c7bc0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:14.577239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e10ea6-04c7-468e-a87b-f8c2a7c7bc0d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:14.578554 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1392] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:14.584649 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676233c5-1c6b-459c-b987-28966eca2fac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:14.585088 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676233c5-1c6b-459c-b987-28966eca2fac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:14.585210 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676233c5-1c6b-459c-b987-28966eca2fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:14.603898 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-676233c5-1c6b-459c-b987-28966eca2fac None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:14.604978 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-676233c5-1c6b-459c-b987-28966eca2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:14.613193 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 704/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:14.659292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e10ea6-04c7-468e-a87b-f8c2a7c7bc0d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:14.674732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e10ea6-04c7-468e-a87b-f8c2a7c7bc0d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:14.680746 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:14.681388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:14.681670 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:14.681945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:14.690415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:14.692032 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9a533-3b46-4d58-b405-30834a3d12c3 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:14.702529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:16.826975 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1dc6c-0124-443d-94c4-0c535068f72b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.827538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1dc6c-0124-443d-94c4-0c535068f72b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.827538 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1dc6c-0124-443d-94c4-0c535068f72b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.828936 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 705/1395] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:16.834802 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07f5a-9150-42d4-9bdf-22f88f98a380 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.835213 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07f5a-9150-42d4-9bdf-22f88f98a380 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.835360 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a07f5a-9150-42d4-9bdf-22f88f98a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.841548 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=52 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:16.855532 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1a07f5a-9150-42d4-9bdf-22f88f98a380 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:16.857033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1a07f5a-9150-42d4-9bdf-22f88f98a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:16.865762 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:16.910400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a3a25-46b5-47ed-a233-d05b6c3282af None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.910400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a3a25-46b5-47ed-a233-d05b6c3282af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.910615 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a3a25-46b5-47ed-a233-d05b6c3282af None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.912548 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 706/1397] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:16.917732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.918021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.918296 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.922434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a070b-762c-477b-b5e7-50748afeeb47 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.922720 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a070b-762c-477b-b5e7-50748afeeb47 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.922992 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a070b-762c-477b-b5e7-50748afeeb47 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.924325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 707/1398] 10.222.0.45 () {54 vars in 962 bytes} [Wed May 25 13:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:45:16.929581 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265fe8-098a-4835-b406-bacafcb4b476 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:16.929864 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265fe8-098a-4835-b406-bacafcb4b476 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:16.930138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265fe8-098a-4835-b406-bacafcb4b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:16.937485 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:16.938675 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:16.946788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1399] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:16.978993 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9a265fe8-098a-4835-b406-bacafcb4b476 None None] MFA Rules not processed for user `593fe462b2d947259086a4f98a043752`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:16.980154 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9a265fe8-098a-4835-b406-bacafcb4b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:16.990763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:17.002513 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073f03d-d18a-4cfc-8a9e-d55107c01960 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:17.008186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:17.008933 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.010221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.010221 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.018122 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:17.019835 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:17.029757 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1400] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:45:17.042085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 708/1401] 10.222.0.45 () {58 vars in 1059 bytes} [Wed May 25 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:45:17.049749 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:17.059428 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13b3b1-26b8-49da-a714-9fe53d522a63 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:17.063160 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '593fe462b2d947259086a4f98a043752', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:17.063871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.064277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.064624 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.074280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:17.075792 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17616-fe79-416b-af74-0b889884855c service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:17.085569 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:17.626098 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ef45-0626-43ab-81e2-3780bb40d463 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.626395 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ef45-0626-43ab-81e2-3780bb40d463 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.626676 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ef45-0626-43ab-81e2-3780bb40d463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.676957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bac0ef45-0626-43ab-81e2-3780bb40d463 None None] MFA Rules not processed for user `316eec64cd5f4184a0b97c946df81a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:17.678147 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bac0ef45-0626-43ab-81e2-3780bb40d463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:17.741384 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:17.747022 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 709/1403] 10.222.0.45 () {58 vars in 1009 bytes} [Wed May 25 13:45:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:17.792907 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4261-395c-45c5-8682-9f11b5c7d084 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.793278 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4261-395c-45c5-8682-9f11b5c7d084 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.793621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b4261-395c-45c5-8682-9f11b5c7d084 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.795115 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1404] 10.222.0.45 () {54 vars in 942 bytes} [Wed May 25 13:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:17.801738 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aad8a3-9027-410e-8d7b-c918e2832720 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.802102 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aad8a3-9027-410e-8d7b-c918e2832720 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.802390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aad8a3-9027-410e-8d7b-c918e2832720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.852809 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-21aad8a3-9027-410e-8d7b-c918e2832720 None None] MFA Rules not processed for user `644184b8dbe74a2e9704ee090c465b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:17.854021 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-21aad8a3-9027-410e-8d7b-c918e2832720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:17.925392 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 710/1405] 10.222.0.45 () {58 vars in 1039 bytes} [Wed May 25 13:45:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:17.934848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b4261-395c-45c5-8682-9f11b5c7d084 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:17.948808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b4261-395c-45c5-8682-9f11b5c7d084 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:17.953688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644184b8dbe74a2e9704ee090c465b57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:17.954320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:17.954603 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:17.954877 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:17.963310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:17.964923 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3c30e-d74b-4c7a-b31a-d3d891d894c5 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:17.972913 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1406] 10.222.0.45 () {60 vars in 1384 bytes} [Wed May 25 13:45:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 13:45:19.300308 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ce4a-0a57-4083-a4f7-454ad2eb3879 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.300865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ce4a-0a57-4083-a4f7-454ad2eb3879 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.300865 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ce4a-0a57-4083-a4f7-454ad2eb3879 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.302414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 711/1407] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:19.309734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c505-effe-4a40-acac-30256b4c60ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.310276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c505-effe-4a40-acac-30256b4c60ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.310276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c505-effe-4a40-acac-30256b4c60ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.331946 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e396c505-effe-4a40-acac-30256b4c60ab None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:19.333539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e396c505-effe-4a40-acac-30256b4c60ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:19.351981 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:19.389348 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ce4a-0a57-4083-a4f7-454ad2eb3879 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:19.405062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ce4a-0a57-4083-a4f7-454ad2eb3879 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:19.410583 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:19.416493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.416493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.416493 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.424949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:19.427515 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7b0d9-b303-418a-a300-9b1b5ce86156 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:19.438137 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 712/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:29.326172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:29.433024 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=50 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:41.455228 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228b49-853c-44cb-9bcb-f66f9cd7e113 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:41.456394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228b49-853c-44cb-9bcb-f66f9cd7e113 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:41.456394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228b49-853c-44cb-9bcb-f66f9cd7e113 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:41.457991 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1410] 10.222.0.45 () {54 vars in 918 bytes} [Wed May 25 13:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:41.462988 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9582-7d40-4699-93bd-ebed4428c3df None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:41.463468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9582-7d40-4699-93bd-ebed4428c3df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:41.463579 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9582-7d40-4699-93bd-ebed4428c3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:41.482243 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d85d9582-7d40-4699-93bd-ebed4428c3df None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:41.483440 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d85d9582-7d40-4699-93bd-ebed4428c3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:41.492153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 713/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Wed May 25 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:41.506185 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f2ea-2235-4e8b-b8aa-348d3f5230be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:41.506476 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f2ea-2235-4e8b-b8aa-348d3f5230be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:41.506752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861f2ea-2235-4e8b-b8aa-348d3f5230be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:41.508172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1412] 10.222.0.45 () {54 vars in 918 bytes} [Wed May 25 13:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:41.512770 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d217ca-89ad-4468-8a03-304bb54fb8c7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:41.513058 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d217ca-89ad-4468-8a03-304bb54fb8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:41.513362 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d217ca-89ad-4468-8a03-304bb54fb8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:41.531606 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-73d217ca-89ad-4468-8a03-304bb54fb8c7 None None] MFA Rules not processed for user `6086ccae239045e5b109e33cf47453a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:41.532807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-73d217ca-89ad-4468-8a03-304bb54fb8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:41.542867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 714/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Wed May 25 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:44.249771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9947-37b8-4145-9a30-f233b37cbf2a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:44.250310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9947-37b8-4145-9a30-f233b37cbf2a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:44.250310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e9947-37b8-4145-9a30-f233b37cbf2a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:44.251693 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:44.257445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69f329-7f91-4c32-af86-6b50e51dd1cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:44.257870 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69f329-7f91-4c32-af86-6b50e51dd1cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:44.257977 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69f329-7f91-4c32-af86-6b50e51dd1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:44.275768 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cd69f329-7f91-4c32-af86-6b50e51dd1cb None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:44.276961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cd69f329-7f91-4c32-af86-6b50e51dd1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:44.285163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:44.327147 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e9947-37b8-4145-9a30-f233b37cbf2a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:44.342078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e9947-37b8-4145-9a30-f233b37cbf2a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:44.348235 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:44.348888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:44.349178 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:44.349457 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:44.358216 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:44.359819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367c706-559f-4586-bf3f-1d8c167052cb service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:44.369276 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:45.331339 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1958-e5e4-47b4-8783-31c6ecdfca89 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.331908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1958-e5e4-47b4-8783-31c6ecdfca89 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.331908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1958-e5e4-47b4-8783-31c6ecdfca89 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.333375 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:45.338432 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f29c0-604c-4322-bb31-58fb4a03079f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.338815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f29c0-604c-4322-bb31-58fb4a03079f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.338995 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f29c0-604c-4322-bb31-58fb4a03079f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.358464 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-846f29c0-604c-4322-bb31-58fb4a03079f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:45.359681 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-846f29c0-604c-4322-bb31-58fb4a03079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:45.367540 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:45.542454 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1958-e5e4-47b4-8783-31c6ecdfca89 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:45.556792 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1958-e5e4-47b4-8783-31c6ecdfca89 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:45.562160 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:45.562795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.563082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.563361 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.572478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:45.574060 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168031-4ec5-4993-8dea-2c61ed27a41b service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:45.584095 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 717/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:46.507237 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bd36-9dff-4c31-99f0-7cbea96c16ed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:46.507772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bd36-9dff-4c31-99f0-7cbea96c16ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:46.507772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93bd36-9dff-4c31-99f0-7cbea96c16ed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:46.509203 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:46.514205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ce9b7-723d-4e2d-ab6c-808637494474 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:46.514585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ce9b7-723d-4e2d-ab6c-808637494474 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:46.514762 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ce9b7-723d-4e2d-ab6c-808637494474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:46.533182 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ed3ce9b7-723d-4e2d-ab6c-808637494474 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:46.534582 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed3ce9b7-723d-4e2d-ab6c-808637494474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:46.542205 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:46.691742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93bd36-9dff-4c31-99f0-7cbea96c16ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:46.706022 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93bd36-9dff-4c31-99f0-7cbea96c16ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:46.711319 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:46.711977 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:46.712264 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:46.712537 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:46.720596 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:46.722405 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccb390-6e06-4665-bacd-ab6c8019cfd6 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:46.732251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:47.632374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f959e-fd54-4a58-a2b0-35211b0c0e11 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.632934 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f959e-fd54-4a58-a2b0-35211b0c0e11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.633040 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f959e-fd54-4a58-a2b0-35211b0c0e11 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.634378 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:47.639244 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533edc6e-7295-4eae-876e-add8bf8447f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.639719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533edc6e-7295-4eae-876e-add8bf8447f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.639886 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533edc6e-7295-4eae-876e-add8bf8447f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.658361 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-533edc6e-7295-4eae-876e-add8bf8447f9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:47.660201 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-533edc6e-7295-4eae-876e-add8bf8447f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:47.668403 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:47.821803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f959e-fd54-4a58-a2b0-35211b0c0e11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:47.836789 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f959e-fd54-4a58-a2b0-35211b0c0e11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:47.842078 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:47.842716 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.843003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.843274 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.851903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:47.853498 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ca181-f3dc-469b-aa02-4c87a02cc1ec service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:47.863175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 720/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:48.756883 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd743b-33c0-4880-b42f-939c8338d8b0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:48.757425 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd743b-33c0-4880-b42f-939c8338d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:48.757521 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd743b-33c0-4880-b42f-939c8338d8b0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:48.758872 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:48.763696 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3c8b-6d84-460f-a342-1fe99e1e62ca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:48.764167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3c8b-6d84-460f-a342-1fe99e1e62ca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:48.764265 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3c8b-6d84-460f-a342-1fe99e1e62ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:48.782554 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a9ea3c8b-6d84-460f-a342-1fe99e1e62ca None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:48.783832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a9ea3c8b-6d84-460f-a342-1fe99e1e62ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:48.791497 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:48.944123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd743b-33c0-4880-b42f-939c8338d8b0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:48.958728 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd743b-33c0-4880-b42f-939c8338d8b0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:48.964153 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:48.964788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:48.965073 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:48.965350 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:48.973545 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:48.975212 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd0d6-ae44-4ac6-9ce1-096adda6e98b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:48.985078 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:49.840699 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8822-2e90-48ac-9325-b393d19c47d3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:49.841290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8822-2e90-48ac-9325-b393d19c47d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:49.841290 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8822-2e90-48ac-9325-b393d19c47d3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:49.842642 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:49.848458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4169e-52f6-48eb-80df-67e33f0edc49 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:49.848895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4169e-52f6-48eb-80df-67e33f0edc49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:49.849018 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4169e-52f6-48eb-80df-67e33f0edc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:49.867019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-63a4169e-52f6-48eb-80df-67e33f0edc49 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:49.868274 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63a4169e-52f6-48eb-80df-67e33f0edc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:49.875862 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:50.025267 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8822-2e90-48ac-9325-b393d19c47d3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:50.039816 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8822-2e90-48ac-9325-b393d19c47d3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:50.045053 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:50.045677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:50.045958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:50.046232 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:50.054528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:50.056121 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6e094-e19d-4718-bc45-e6b30180ce38 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:50.066438 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 723/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:50.928114 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98b1e6-4841-4f3f-a20b-32eb347568ff None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:50.928702 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98b1e6-4841-4f3f-a20b-32eb347568ff None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:50.928702 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98b1e6-4841-4f3f-a20b-32eb347568ff None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:50.930033 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:50.935013 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c95a4-e3c7-4410-9b77-e09d191d58bf None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:50.935403 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c95a4-e3c7-4410-9b77-e09d191d58bf None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:50.935587 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c95a4-e3c7-4410-9b77-e09d191d58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:50.954016 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c3c95a4-e3c7-4410-9b77-e09d191d58bf None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:50.955866 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c3c95a4-e3c7-4410-9b77-e09d191d58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:50.963527 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:51.114299 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98b1e6-4841-4f3f-a20b-32eb347568ff None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:51.129901 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98b1e6-4841-4f3f-a20b-32eb347568ff None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:51.135374 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:51.136041 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:51.136328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:51.136606 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:51.144734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:51.146326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b1497-9150-4133-adaf-2fcf86c09e6b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:51.155855 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:52.004276 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f642b-252e-4d37-8ccf-2f06bba8775e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.004818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f642b-252e-4d37-8ccf-2f06bba8775e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.004818 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f642b-252e-4d37-8ccf-2f06bba8775e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.006260 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:52.011277 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484495-3bcc-4dcd-87cb-f9ed452b8297 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.011671 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484495-3bcc-4dcd-87cb-f9ed452b8297 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.011898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0484495-3bcc-4dcd-87cb-f9ed452b8297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.030381 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0484495-3bcc-4dcd-87cb-f9ed452b8297 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:52.031580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0484495-3bcc-4dcd-87cb-f9ed452b8297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:52.040238 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:52.190946 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f642b-252e-4d37-8ccf-2f06bba8775e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:52.204894 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f642b-252e-4d37-8ccf-2f06bba8775e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:52.211158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:52.211779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.212085 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.212360 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.220318 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:52.222510 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafc29d-c472-4213-b181-ddad58020144 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:52.232170 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 726/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:53.089472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197600-adf3-4713-9c14-2bead4fa2c79 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:53.090054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197600-adf3-4713-9c14-2bead4fa2c79 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:53.090054 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197600-adf3-4713-9c14-2bead4fa2c79 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:53.091419 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:53.096295 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4cd03-6984-45e6-a613-77e8168c325f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:53.096740 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4cd03-6984-45e6-a613-77e8168c325f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:53.096994 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4cd03-6984-45e6-a613-77e8168c325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:53.115478 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16f4cd03-6984-45e6-a613-77e8168c325f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:53.116705 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16f4cd03-6984-45e6-a613-77e8168c325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:53.124650 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:53.272430 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c197600-adf3-4713-9c14-2bead4fa2c79 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:53.286499 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c197600-adf3-4713-9c14-2bead4fa2c79 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:53.291854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:53.292530 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:53.292814 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:53.293092 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:53.303025 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:53.304653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9a203-ebf9-4f63-a949-430f75ef7706 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:53.313893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:55.402844 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e77a-d95e-4d87-bf2b-ad8c5e123f11 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:55.403301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e77a-d95e-4d87-bf2b-ad8c5e123f11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:55.403406 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2e77a-d95e-4d87-bf2b-ad8c5e123f11 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:55.404802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:55.410943 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d5d-8608-4e50-bcae-f47d71b3bce1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:55.411366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d5d-8608-4e50-bcae-f47d71b3bce1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:55.411498 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e58d5d-8608-4e50-bcae-f47d71b3bce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:55.429271 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0e58d5d-8608-4e50-bcae-f47d71b3bce1 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:55.430453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0e58d5d-8608-4e50-bcae-f47d71b3bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:55.438091 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:55.477031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2e77a-d95e-4d87-bf2b-ad8c5e123f11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:55.488580 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2e77a-d95e-4d87-bf2b-ad8c5e123f11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:55.492169 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:55.492803 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:55.493082 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:55.493354 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:55.503079 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:55.504531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40481-5001-47ec-ba62-6dd71e6d0266 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:55.514434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 729/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:57.928454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ba8fb-e3f7-47b8-a388-0496bf6318ed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:57.929022 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ba8fb-e3f7-47b8-a388-0496bf6318ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:57.929022 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ba8fb-e3f7-47b8-a388-0496bf6318ed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:57.930366 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:57.935703 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98313c-0486-4ddf-a9b5-3ac7139b424e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:57.936245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98313c-0486-4ddf-a9b5-3ac7139b424e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:57.936344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98313c-0486-4ddf-a9b5-3ac7139b424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:57.955288 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc98313c-0486-4ddf-a9b5-3ac7139b424e None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:57.956517 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc98313c-0486-4ddf-a9b5-3ac7139b424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:57.964212 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:58.001893 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ba8fb-e3f7-47b8-a388-0496bf6318ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:58.012495 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ba8fb-e3f7-47b8-a388-0496bf6318ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:58.015962 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:58.016595 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:58.016876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:58.017808 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:58.026665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:58.028136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291379e3-7283-4db8-a702-b39ae687f4d6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:58.037945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:00.536660 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c9a6-f793-473c-a392-af9bc9e765b1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:00.537218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c9a6-f793-473c-a392-af9bc9e765b1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:00.537218 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c9a6-f793-473c-a392-af9bc9e765b1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:00.538573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:00.543898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9046-a7af-4b54-8c5a-6df19ee26cad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:00.544332 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9046-a7af-4b54-8c5a-6df19ee26cad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:00.544487 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e9046-a7af-4b54-8c5a-6df19ee26cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:00.562650 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ac1e9046-a7af-4b54-8c5a-6df19ee26cad None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:00.564263 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac1e9046-a7af-4b54-8c5a-6df19ee26cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:00.571942 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:46:00.611270 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586c9a6-f793-473c-a392-af9bc9e765b1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:00.621867 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586c9a6-f793-473c-a392-af9bc9e765b1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:00.625344 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:00.625974 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:00.626305 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:00.626585 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:00.635427 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:00.636883 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83a2b2-9cc0-477e-9a0a-840f7a93d97e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:00.646556 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 732/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:03.033172 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89455-9e82-47f5-a0e3-3a5e102b5e0c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:03.033777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89455-9e82-47f5-a0e3-3a5e102b5e0c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:03.033777 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89455-9e82-47f5-a0e3-3a5e102b5e0c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:03.035157 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:03.040345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced1a3a-dace-4394-affe-4b9a5f31d183 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:03.040764 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced1a3a-dace-4394-affe-4b9a5f31d183 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:03.040899 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced1a3a-dace-4394-affe-4b9a5f31d183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:03.058995 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bced1a3a-dace-4394-affe-4b9a5f31d183 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:03.060229 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bced1a3a-dace-4394-affe-4b9a5f31d183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:03.068733 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:03.106578 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89455-9e82-47f5-a0e3-3a5e102b5e0c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:03.117895 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89455-9e82-47f5-a0e3-3a5e102b5e0c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:03.121614 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:03.122245 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:03.122529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:03.122803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:03.132783 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:03.134259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb3081-6012-454a-9cf4-5bfe9b13c386 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:03.143857 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:46:05.393542 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f11c-3a85-483c-ba13-b491b86200f8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:05.394030 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f11c-3a85-483c-ba13-b491b86200f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:05.394144 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f11c-3a85-483c-ba13-b491b86200f8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:05.395473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:05.400894 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c91cd-7c31-4bbf-8035-1442ffd4090d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:05.401326 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c91cd-7c31-4bbf-8035-1442ffd4090d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:05.401448 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c91cd-7c31-4bbf-8035-1442ffd4090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:05.419660 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-131c91cd-7c31-4bbf-8035-1442ffd4090d None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:05.420898 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-131c91cd-7c31-4bbf-8035-1442ffd4090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:05.428773 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:05.467076 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f11c-3a85-483c-ba13-b491b86200f8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:05.478601 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f11c-3a85-483c-ba13-b491b86200f8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:05.482087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:05.482719 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:05.483003 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:05.483280 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:05.492404 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:05.493823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23114b-29aa-4493-aaa4-46be74328bd6 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:05.503291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 735/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:07.793320 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb825fb-bf18-4b20-9b02-40eb09f160cf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:07.793899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb825fb-bf18-4b20-9b02-40eb09f160cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:07.793899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb825fb-bf18-4b20-9b02-40eb09f160cf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:07.795266 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:07.800599 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429255-ed75-4226-a319-95dc9cf4f019 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:07.801032 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429255-ed75-4226-a319-95dc9cf4f019 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:07.801148 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429255-ed75-4226-a319-95dc9cf4f019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:07.819976 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ff429255-ed75-4226-a319-95dc9cf4f019 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:07.821175 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ff429255-ed75-4226-a319-95dc9cf4f019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:07.830241 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:07.867914 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb825fb-bf18-4b20-9b02-40eb09f160cf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:07.878994 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb825fb-bf18-4b20-9b02-40eb09f160cf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:07.882422 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:07.883045 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:07.883324 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:07.883597 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:07.893081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:07.894506 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9e46a-8f82-47ad-b6d4-a10eb9f95b86 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:07.903621 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:10.193856 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080277d-750c-4b89-b71b-f0d417ec9fd0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:10.194447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080277d-750c-4b89-b71b-f0d417ec9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:10.194447 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080277d-750c-4b89-b71b-f0d417ec9fd0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:10.195893 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:10.201407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19f720-6bb9-469f-911a-954c5d505233 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:10.201842 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19f720-6bb9-469f-911a-954c5d505233 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:10.201958 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19f720-6bb9-469f-911a-954c5d505233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:10.208645 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=70 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:10.221297 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b19f720-6bb9-469f-911a-954c5d505233 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:10.222494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b19f720-6bb9-469f-911a-954c5d505233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:10.230555 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:10.268528 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080277d-750c-4b89-b71b-f0d417ec9fd0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:10.273888 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=73 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:10.280158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080277d-750c-4b89-b71b-f0d417ec9fd0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:10.283708 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:10.284379 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:10.284665 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:10.284942 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:10.293749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:10.295171 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8207eaf-5829-48a6-b715-d5ee9457141d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:10.304573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 738/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:12.524664 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:12.525224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:12.525224 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:12.526583 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:12.532613 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6577ed-a69e-4b86-9664-01bdfe76d94b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:12.533044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6577ed-a69e-4b86-9664-01bdfe76d94b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:12.533044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6577ed-a69e-4b86-9664-01bdfe76d94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:12.550819 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6f6577ed-a69e-4b86-9664-01bdfe76d94b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:12.552044 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6f6577ed-a69e-4b86-9664-01bdfe76d94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:12.559677 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:12.597811 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:12.599510 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:12.613308 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eec429-adf8-41a0-9972-efa09d6fd307 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:12.616876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:12.617512 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:12.617798 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:12.618077 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:12.626944 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:12.628420 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a76ed-b477-4d92-8ac0-c6aa8659e05f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:12.637694 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:14.915620 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8645c1-dbea-4218-b3db-2b8dbcd739e4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:14.916217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8645c1-dbea-4218-b3db-2b8dbcd739e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:14.916217 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8645c1-dbea-4218-b3db-2b8dbcd739e4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:14.917590 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:14.922797 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf458-d6fa-4bac-9827-8ec02025727c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:14.923239 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf458-d6fa-4bac-9827-8ec02025727c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:14.923355 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf458-d6fa-4bac-9827-8ec02025727c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:14.941300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-02fcf458-d6fa-4bac-9827-8ec02025727c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:14.942490 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-02fcf458-d6fa-4bac-9827-8ec02025727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:14.951352 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:14.988873 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8645c1-dbea-4218-b3db-2b8dbcd739e4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:14.999494 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8645c1-dbea-4218-b3db-2b8dbcd739e4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:15.003543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:15.004207 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:15.004496 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:15.004780 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:15.013573 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:15.014996 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cef22-0444-40f2-8e37-4fb301223776 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:15.025158 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 741/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:17.236406 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f00d-4864-411c-808b-4f0928860325 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:17.237017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f00d-4864-411c-808b-4f0928860325 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:17.237017 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f00d-4864-411c-808b-4f0928860325 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:17.238337 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:17.243569 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060782e5-428f-454e-b12a-4f523d8ab24b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:17.243991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060782e5-428f-454e-b12a-4f523d8ab24b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:17.244167 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060782e5-428f-454e-b12a-4f523d8ab24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:17.262135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-060782e5-428f-454e-b12a-4f523d8ab24b None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:17.263325 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-060782e5-428f-454e-b12a-4f523d8ab24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:17.271884 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:17.307643 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f00d-4864-411c-808b-4f0928860325 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:17.318965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f00d-4864-411c-808b-4f0928860325 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:17.322547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:17.323180 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:17.323466 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:17.323752 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:17.332911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:17.334345 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cbb86-5f80-4e44-86fe-db66b936b70d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:17.343587 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:27.257247 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=17 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:27.339481 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=17 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:50.446284 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e097-520e-4cfa-be20-a23f4fa5defc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:50.446869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e097-520e-4cfa-be20-a23f4fa5defc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:50.446869 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e097-520e-4cfa-be20-a23f4fa5defc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:50.448244 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 743/1471] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:50.454303 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901d5-35dd-4957-aa0d-81177c69db70 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:50.454745 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901d5-35dd-4957-aa0d-81177c69db70 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:50.454885 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901d5-35dd-4957-aa0d-81177c69db70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:50.473611 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c0901d5-35dd-4957-aa0d-81177c69db70 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:50.474789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c0901d5-35dd-4957-aa0d-81177c69db70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:50.483280 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:50.530433 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08603-d1b7-44c1-81c4-1e534d2e1850 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:50.530751 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08603-d1b7-44c1-81c4-1e534d2e1850 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:50.531031 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f08603-d1b7-44c1-81c4-1e534d2e1850 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:50.532469 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 744/1473] 10.222.0.45 () {54 vars in 962 bytes} [Wed May 25 13:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:50.537268 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95630e-751a-473c-a180-5e192de85600 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:50.537556 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95630e-751a-473c-a180-5e192de85600 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:50.537834 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95630e-751a-473c-a180-5e192de85600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:50.555539 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1a95630e-751a-473c-a180-5e192de85600 None None] MFA Rules not processed for user `593fe462b2d947259086a4f98a043752`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:50.556742 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1a95630e-751a-473c-a180-5e192de85600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:50.563695 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Wed May 25 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:50.570678 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08603-d1b7-44c1-81c4-1e534d2e1850 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:50.581938 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f08603-d1b7-44c1-81c4-1e534d2e1850 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:50.585430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '593fe462b2d947259086a4f98a043752', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:50.586051 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:50.586331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:50.586602 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:50.596062 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:50.597474 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85bc25-8622-4a64-8fa6-79b0de11db8b service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:50.606958 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 745/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:52.899395 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f68d-b8ca-4eb2-bafa-9690f0b6f295 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:52.899982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f68d-b8ca-4eb2-bafa-9690f0b6f295 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:52.899982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f68d-b8ca-4eb2-bafa-9690f0b6f295 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:52.901348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1476] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:52.907200 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e9e7-c801-4854-8f80-93410e594b1f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:52.907644 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e9e7-c801-4854-8f80-93410e594b1f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:52.907761 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e9e7-c801-4854-8f80-93410e594b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:52.926177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c274e9e7-c801-4854-8f80-93410e594b1f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:52.927365 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c274e9e7-c801-4854-8f80-93410e594b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:52.935094 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 746/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:52.958333 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f68d-b8ca-4eb2-bafa-9690f0b6f295 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:52.971062 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f68d-b8ca-4eb2-bafa-9690f0b6f295 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:52.975734 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:52.976410 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:52.976722 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:52.977000 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:52.977735 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:52.979522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:46:52.979522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) May 25 13:46:52.979522 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: May 25 13:46:52.980719 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bf30-8a60-4f8d-b88f-3f192a420c81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:53.000580 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Wed May 25 13:46:52 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:46:55.253231 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91eb45-8df0-4465-a53b-57b3adc0b09f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:55.253810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91eb45-8df0-4465-a53b-57b3adc0b09f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:55.253810 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91eb45-8df0-4465-a53b-57b3adc0b09f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:55.255168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 747/1479] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:55.260373 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7636a-26f9-47f1-a0ea-d838589b5900 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:55.260796 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7636a-26f9-47f1-a0ea-d838589b5900 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:55.260945 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7636a-26f9-47f1-a0ea-d838589b5900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:55.279610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d7c7636a-26f9-47f1-a0ea-d838589b5900 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:55.280861 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7c7636a-26f9-47f1-a0ea-d838589b5900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:55.288557 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:46:55.332128 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91eb45-8df0-4465-a53b-57b3adc0b09f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:55.346634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91eb45-8df0-4465-a53b-57b3adc0b09f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:55.352434 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:55.353087 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:55.353374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:55.353652 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:55.362216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:55.363785 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b7d80-af38-4eef-aca7-a8dfb78affdf service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:55.373812 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 748/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:57.776417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39fa0-7034-4560-a5e4-c70b9663cf24 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:57.776947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39fa0-7034-4560-a5e4-c70b9663cf24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:57.776947 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39fa0-7034-4560-a5e4-c70b9663cf24 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:57.778272 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1482] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:57.784558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c17e1-5971-491b-bb26-f2af77ee298f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:57.784998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c17e1-5971-491b-bb26-f2af77ee298f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:57.785103 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c17e1-5971-491b-bb26-f2af77ee298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:57.802682 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-646c17e1-5971-491b-bb26-f2af77ee298f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:57.803871 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-646c17e1-5971-491b-bb26-f2af77ee298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:57.811742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 749/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:57.849066 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39fa0-7034-4560-a5e4-c70b9663cf24 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:57.859848 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39fa0-7034-4560-a5e4-c70b9663cf24 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:57.863310 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:57.863966 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:57.864256 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:57.864534 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:57.873328 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:57.874763 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8240-0192-4635-8f1c-663e597dae73 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:57.884348 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:47:00.186923 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88ea0-ee65-4e38-949a-7de9ab57fd6f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:00.187544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88ea0-ee65-4e38-949a-7de9ab57fd6f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:00.187544 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88ea0-ee65-4e38-949a-7de9ab57fd6f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:00.188992 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 750/1485] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:47:00.194417 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fbb7a-5a19-496e-8f22-8472051559b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:00.194856 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fbb7a-5a19-496e-8f22-8472051559b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:00.195020 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fbb7a-5a19-496e-8f22-8472051559b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:00.212665 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f8fbb7a-5a19-496e-8f22-8472051559b9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:47:00.213853 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f8fbb7a-5a19-496e-8f22-8472051559b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:47:00.221727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:47:00.260814 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d88ea0-ee65-4e38-949a-7de9ab57fd6f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:47:00.271468 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d88ea0-ee65-4e38-949a-7de9ab57fd6f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:47:00.274984 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:47:00.275610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:00.275892 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:00.276198 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:00.285196 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:47:00.286610 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc018d3e-dee9-4674-8657-686ec636b7ca service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:47:00.296843 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 751/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:47:02.823716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b763e7-5b52-41fb-a5f8-3379834d65f1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:02.824301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b763e7-5b52-41fb-a5f8-3379834d65f1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:02.824301 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b763e7-5b52-41fb-a5f8-3379834d65f1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:02.826265 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1488] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:47:02.831525 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203cc7-be98-40b4-8f8d-208566546222 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:02.832177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203cc7-be98-40b4-8f8d-208566546222 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:02.832177 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203cc7-be98-40b4-8f8d-208566546222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:02.850374 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c203cc7-be98-40b4-8f8d-208566546222 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:47:02.851552 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c203cc7-be98-40b4-8f8d-208566546222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:47:02.860168 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 752/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:47:02.900031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b763e7-5b52-41fb-a5f8-3379834d65f1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:47:02.914049 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b763e7-5b52-41fb-a5f8-3379834d65f1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:47:02.917533 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:47:02.918167 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:02.918451 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:02.918725 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:02.928544 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:47:02.929992 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099aedf-3428-4e03-b3c6-0a9cb69a9514 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:47:02.939376 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:47:05.365220 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80094680-82b1-4145-ba32-17c68c9dd2f9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:05.365714 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80094680-82b1-4145-ba32-17c68c9dd2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:05.365827 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80094680-82b1-4145-ba32-17c68c9dd2f9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:05.367227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 753/1491] 10.222.0.45 () {54 vars in 926 bytes} [Wed May 25 13:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:47:05.373021 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570bf5c-3bae-4e21-a8e7-3cf98d11edd9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:05.373516 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570bf5c-3bae-4e21-a8e7-3cf98d11edd9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:05.373631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570bf5c-3bae-4e21-a8e7-3cf98d11edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:05.391355 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2570bf5c-3bae-4e21-a8e7-3cf98d11edd9 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:47:05.392572 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2570bf5c-3bae-4e21-a8e7-3cf98d11edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:47:05.400625 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Wed May 25 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:47:05.439886 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80094680-82b1-4145-ba32-17c68c9dd2f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:47:05.451345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80094680-82b1-4145-ba32-17c68c9dd2f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:47:05.454903 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:47:05.455563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:47:05.455841 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:47:05.456135 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:47:05.464961 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:47:05.466990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df305bb-4401-42cc-aad6-d19b9fda2bce service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:47:05.476352 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 754/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:47:15.386081 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=33 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:47:15.472084 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=33 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:45.192865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.193446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.193446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.211159 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:45.212404 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:45.220075 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1494] 10.222.0.45 () {58 vars in 957 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:45.230068 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.230562 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.230674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.248531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:45.249697 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:45.257323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 755/1495] 10.222.0.45 () {58 vars in 957 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:45.261845 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.271233 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c607-79ed-4f78-a3be-8fdb748e76ea None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.275867 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.276547 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.276828 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.277101 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.277819 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:48:45.278125 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.279150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.279150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) May 25 13:48:45.279150 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: May 25 13:48:45.280067 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.292148 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Wed May 25 13:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:48:45.300260 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.309322 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8fb4-c6c0-4b34-bc2b-8143570b7773 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.313757 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.314393 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.314673 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.314944 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.315625 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.317414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.317414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 13:48:45.317414 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: May 25 13:48:45.318595 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.358511 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 756/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:48:45.362784 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.372795 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f56ff5-b83e-4530-b698-029c7736952d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.428014 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.428680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.428963 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.429241 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.430454 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.431511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.431511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) May 25 13:48:45.431511 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: May 25 13:48:45.432771 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.453156 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:48:45.457010 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.467550 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb50824-49ee-456d-8132-06faa3d11c76 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.472331 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.472982 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.473263 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.473540 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.474259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.475259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.475259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 13:48:45.475259 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: May 25 13:48:45.476153 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.489036 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 757/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Wed May 25 13:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 13:48:45.493378 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.503098 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960738bb-edd8-484e-87a4-47f13f137f99 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.507716 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.508566 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.508854 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.509133 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.509858 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.510876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.510876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) May 25 13:48:45.510876 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: May 25 13:48:45.511765 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fd62a-9221-4e9a-8e35-18e41ea8662b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.525452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Wed May 25 13:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:48:45.529486 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.539248 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d99cc6-e13e-4b9a-aec5-3458a8430a45 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.544473 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.545402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.545685 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.545990 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] PATH_INFO: `/v3/projects/fd4f02455e714f019ebc314533c5f80a/users/844e5324c43f4b7790e6f5b19679ff3e/roles/eda7e112a67a4f12993c9af8f1f3b587` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.560432 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4f02455e714f019ebc314533c5f80a, user_id=844e5324c43f4b7790e6f5b19679ff3e, role_id=eda7e112a67a4f12993c9af8f1f3b587)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.561461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:45.561461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 13:48:45.561461 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: May 25 13:48:45.563402 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ae99f-d565-4d4a-8282-3b2ca6538c8f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.577112 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 758/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Wed May 25 13:48:45 2022] PUT /identity/v3/projects/fd4f02455e714f019ebc314533c5f80a/users/844e5324c43f4b7790e6f5b19679ff3e/roles/eda7e112a67a4f12993c9af8f1f3b587 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:45.588732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582a09-cf8c-4238-ac14-63137ad2de0a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.589031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582a09-cf8c-4238-ac14-63137ad2de0a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.589318 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582a09-cf8c-4238-ac14-63137ad2de0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.615631 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e6582a09-cf8c-4238-ac14-63137ad2de0a None None] MFA Rules not processed for user `844e5324c43f4b7790e6f5b19679ff3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:45.616865 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6582a09-cf8c-4238-ac14-63137ad2de0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:45.680953 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1502] 10.222.0.45 () {58 vars in 957 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:45.693962 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151af7c-9904-4381-91f0-5ef52b604852 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.694255 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151af7c-9904-4381-91f0-5ef52b604852 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.694531 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151af7c-9904-4381-91f0-5ef52b604852 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.696637 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 759/1503] 10.222.0.45 () {54 vars in 963 bytes} [Wed May 25 13:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:48:45.701610 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78d37-8ffa-494f-8622-ccd7c98f3730 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.701899 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78d37-8ffa-494f-8622-ccd7c98f3730 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.702176 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c78d37-8ffa-494f-8622-ccd7c98f3730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.720302 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d5c78d37-8ffa-494f-8622-ccd7c98f3730 None None] MFA Rules not processed for user `3e02e0b0eac04b51970fa87a0df61e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:45.721462 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d5c78d37-8ffa-494f-8622-ccd7c98f3730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:45.766019 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:45.775488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Wed May 25 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:45.889851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151af7c-9904-4381-91f0-5ef52b604852 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:45.901400 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151af7c-9904-4381-91f0-5ef52b604852 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:45.904989 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e02e0b0eac04b51970fa87a0df61e64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:45.905619 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.905904 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.906183 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.914323 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:45.915949 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:45.924648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 760/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:48:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:48:45.966879 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd019-fd66-4758-9b53-8589ab0a8c7c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:45.967175 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd019-fd66-4758-9b53-8589ab0a8c7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:45.967458 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafd019-fd66-4758-9b53-8589ab0a8c7c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:45.968782 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1506] 10.222.0.45 () {52 vars in 829 bytes} [Wed May 25 13:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:48:45.972640 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:46.034287 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2a2fa-b0cf-48f5-b604-3c9525b3a812 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:46.038835 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:46.039831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:46.040142 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:46.040424 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] PATH_INFO: `/v3/users/844e5324c43f4b7790e6f5b19679ff3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:46.041619 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=844e5324c43f4b7790e6f5b19679ff3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:46.042627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:46.042627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 25 13:48:46.042627 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: May 25 13:48:46.043829 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:46.109358 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844e5324c43f4b7790e6f5b19679ff3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.121345 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844e5324c43f4b7790e6f5b19679ff3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.121987 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844e5324c43f4b7790e6f5b19679ff3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.141070 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844e5324c43f4b7790e6f5b19679ff3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.150963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:46.156791 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844e5324c43f4b7790e6f5b19679ff3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.164364 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 761/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Wed May 25 13:48:45 2022] DELETE /identity/v3/users/844e5324c43f4b7790e6f5b19679ff3e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:46.169007 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:46.169300 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:46.169577 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:46.187590 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:46.188800 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:46.247433 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1508] 10.222.0.45 () {58 vars in 957 bytes} [Wed May 25 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:46.256747 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:46.258799 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:46.314381 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2327-2fd7-4eb7-8bed-e04340115a4a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:46.317919 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:46.318563 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:46.318847 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:46.319125 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:46.328639 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:46.330075 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112627fa-1547-413c-8def-b83e6217c755 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:46.339465 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 762/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:48:46.744314 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:46.745760 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:46.757292 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa077a2-0999-4e41-90f9-c65f1b11dc08 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:46.761998 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47296119d0ef495489c4b5806001b954', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c319a094752b41428929e5f2edeaa06e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:46.762851 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:46.763136 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:46.763415 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] PATH_INFO: `/v3/projects/fd4f02455e714f019ebc314533c5f80a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:46.764627 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4f02455e714f019ebc314533c5f80a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:46.771142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:46.771142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) May 25 13:48:46.771142 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: May 25 13:48:46.772442 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:46.836911 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4f02455e714f019ebc314533c5f80a', 'request_id': 'req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.850574 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4f02455e714f019ebc314533c5f80a', 'request_id': 'req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.857860 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4f02455e714f019ebc314533c5f80a', 'request_id': 'req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.874099 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] The token cache is being invalidate because project fd4f02455e714f019ebc314533c5f80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 13:48:46.874386 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.875012 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba5fb755-1a75-4f4e-af47-bef7ffc5a0f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:46.876709 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Wed May 25 13:48:46 2022] DELETE /identity/v3/projects/fd4f02455e714f019ebc314533c5f80a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:56.335227 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=29 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:56.870982 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:30.948963 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:30.949565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:30.949565 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:30.950908 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:30.955935 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3ccf1-63ab-4da6-9e28-26a4c507882c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:30.956379 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3ccf1-63ab-4da6-9e28-26a4c507882c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:30.956488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3ccf1-63ab-4da6-9e28-26a4c507882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:30.975251 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2bb3ccf1-63ab-4da6-9e28-26a4c507882c None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:30.976489 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2bb3ccf1-63ab-4da6-9e28-26a4c507882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:31.035680 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:31.212163 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:31.214500 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:31.291029 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d022b-e131-48e5-a8a4-f6c4ac422be8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:31.296570 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:31.297216 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:31.297501 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:31.297779 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:31.305742 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:31.307324 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9036717-24ed-4843-96e6-80455c8d2824 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:31.316679 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 764/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:32.203948 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f3e3-43c6-4d43-a44e-561a66579bcb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.204501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f3e3-43c6-4d43-a44e-561a66579bcb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.204501 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600f3e3-43c6-4d43-a44e-561a66579bcb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.205804 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:32.210745 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4791e4-27f8-47fe-a7cc-b93a30e177c0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.211179 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4791e4-27f8-47fe-a7cc-b93a30e177c0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.211291 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4791e4-27f8-47fe-a7cc-b93a30e177c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.229851 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5a4791e4-27f8-47fe-a7cc-b93a30e177c0 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:32.231197 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5a4791e4-27f8-47fe-a7cc-b93a30e177c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:32.239065 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:32.390815 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600f3e3-43c6-4d43-a44e-561a66579bcb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:32.405482 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600f3e3-43c6-4d43-a44e-561a66579bcb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:32.410912 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:32.411564 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.411871 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.412193 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.420825 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:32.422453 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992324df-609a-49e4-b9b6-4671c1dc878b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:32.432406 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:32.458311 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a827d4-fdff-4e22-88d4-4e8beabd427e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.458545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a827d4-fdff-4e22-88d4-4e8beabd427e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.458831 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a827d4-fdff-4e22-88d4-4e8beabd427e None None] PATH_INFO: `/v3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.460180 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 766/1517] 10.222.0.66 () {54 vars in 925 bytes} [Wed May 25 13:50:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:50:32.466452 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb031c-5e5b-4080-a7e0-901c6f524739 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.466812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb031c-5e5b-4080-a7e0-901c6f524739 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.467186 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb031c-5e5b-4080-a7e0-901c6f524739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.487040 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-03fb031c-5e5b-4080-a7e0-901c6f524739 None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:32.488394 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03fb031c-5e5b-4080-a7e0-901c6f524739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:32.555068 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Wed May 25 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:36.435204 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8a901-41ff-4f23-9a40-743c6faf8675 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:36.435674 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8a901-41ff-4f23-9a40-743c6faf8675 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:36.435795 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8a901-41ff-4f23-9a40-743c6faf8675 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:36.437164 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 767/1519] 10.222.0.45 () {54 vars in 960 bytes} [Wed May 25 13:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:36.442106 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcb2e-2172-46ce-a2d6-664857e5955d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:36.442494 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcb2e-2172-46ce-a2d6-664857e5955d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:36.442690 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcb2e-2172-46ce-a2d6-664857e5955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:36.461053 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c1abcb2e-2172-46ce-a2d6-664857e5955d None None] MFA Rules not processed for user `6f6d6866832f456d84536d3cc45e512c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:36.462240 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1abcb2e-2172-46ce-a2d6-664857e5955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:36.471649 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Wed May 25 13:50:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:36.479504 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8a901-41ff-4f23-9a40-743c6faf8675 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:36.493333 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8a901-41ff-4f23-9a40-743c6faf8675 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:36.499138 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6d6866832f456d84536d3cc45e512c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:36.499807 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:36.500126 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:36.500409 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:36.508840 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:36.510460 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a9e96-de67-4901-820e-400b28d0ff16 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:36.520430 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 768/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:43.659739 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a30fd-d7fa-419c-95d0-bb6dd6e78af8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:43.660353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a30fd-d7fa-419c-95d0-bb6dd6e78af8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:43.660353 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a30fd-d7fa-419c-95d0-bb6dd6e78af8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:43.661830 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:43.666860 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fdf8-8b20-4ec4-8712-59405c7aa692 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:43.667301 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fdf8-8b20-4ec4-8712-59405c7aa692 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:43.667421 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84fdf8-8b20-4ec4-8712-59405c7aa692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:43.686073 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ae84fdf8-8b20-4ec4-8712-59405c7aa692 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:43.687245 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ae84fdf8-8b20-4ec4-8712-59405c7aa692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:43.694991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:43.847803 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a30fd-d7fa-419c-95d0-bb6dd6e78af8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:43.862491 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a30fd-d7fa-419c-95d0-bb6dd6e78af8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:43.867781 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:43.868446 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:43.868731 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:43.869008 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:43.877732 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:43.879335 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be729ab-4f49-4ec7-98b8-3c2f36fb725f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:43.888727 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:44.768045 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd690-9d3a-4444-a45e-57626afc58aa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:44.768618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd690-9d3a-4444-a45e-57626afc58aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:44.768618 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afd690-9d3a-4444-a45e-57626afc58aa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:44.769998 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:44.774961 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edb93-3734-418c-b8ee-778fefbe3bac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:44.775388 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edb93-3734-418c-b8ee-778fefbe3bac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:44.775529 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edb93-3734-418c-b8ee-778fefbe3bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:44.793888 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-998edb93-3734-418c-b8ee-778fefbe3bac None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:44.795192 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-998edb93-3734-418c-b8ee-778fefbe3bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:44.802970 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:44.955575 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd690-9d3a-4444-a45e-57626afc58aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:44.969830 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afd690-9d3a-4444-a45e-57626afc58aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:44.975370 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6086ccae239045e5b109e33cf47453a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:44.976037 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:44.976327 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:44.976607 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:44.984991 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:44.986634 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b4dac-63df-4aa6-b8d3-7aab41a1fed8 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:44.996558 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 771/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:49.154812 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:49.155371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:49.155371 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:49.156789 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:49.161881 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934580-b064-4432-98ea-e50688be16d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:49.162281 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934580-b064-4432-98ea-e50688be16d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:49.162445 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934580-b064-4432-98ea-e50688be16d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:49.180603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c1934580-b064-4432-98ea-e50688be16d7 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:49.181824 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1934580-b064-4432-98ea-e50688be16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:49.189749 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:49.229649 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:49.231492 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:49.291055 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b0f7-0cbb-4b05-b56c-8541bd435311 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:49.294723 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:49.295365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:49.295653 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:49.295960 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:49.304922 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:49.306351 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1141b4b-eac2-4736-a847-7d334e678cb5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:49.311313 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:49.316592 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:51.538823 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d778e-8147-44b8-a4de-3120ef129487 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:51.539271 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d778e-8147-44b8-a4de-3120ef129487 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:51.539416 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d778e-8147-44b8-a4de-3120ef129487 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:51.540858 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:51.546915 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92547-2032-4bb7-ae62-9377463ebb9f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:51.547341 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92547-2032-4bb7-ae62-9377463ebb9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:51.547472 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92547-2032-4bb7-ae62-9377463ebb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:51.566057 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b8c92547-2032-4bb7-ae62-9377463ebb9f None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:51.567252 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8c92547-2032-4bb7-ae62-9377463ebb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:51.575688 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:51.611210 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d778e-8147-44b8-a4de-3120ef129487 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:51.622752 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d778e-8147-44b8-a4de-3120ef129487 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:51.626321 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb026adfce894a72b97d750cdbb72efc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:51.626957 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:51.627242 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:51.627520 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:51.636545 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:51.637960 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09165ce-e780-4462-ac42-91fb97b53b83 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:51.647272 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 774/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:55.396253 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:55.396772 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:55.396884 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:55.398163 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:55.403603 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7e02-8801-4ed7-aa6e-6cbcabe12454 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:55.404150 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7e02-8801-4ed7-aa6e-6cbcabe12454 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:55.404273 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7e02-8801-4ed7-aa6e-6cbcabe12454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:55.423390 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8edd7e02-8801-4ed7-aa6e-6cbcabe12454 None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:55.424632 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8edd7e02-8801-4ed7-aa6e-6cbcabe12454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:55.434692 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:55.703259 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:55.704965 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:55.761632 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981fc47-1327-445f-a649-e7fe57653a1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:55.765151 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e02e0b0eac04b51970fa87a0df61e64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:55.765788 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:55.766070 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:55.766342 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:55.775031 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:55.776502 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414eb8a-e5a4-47c8-8e14-00f40505b18e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:55.786407 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:58.033832 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:58.034377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:58.034377 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:58.035802 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:58.041365 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bb0db-9dbc-4e7a-ba52-0757ad213aba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:58.041791 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bb0db-9dbc-4e7a-ba52-0757ad213aba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:58.041929 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bb0db-9dbc-4e7a-ba52-0757ad213aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:58.060270 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a1bb0db-9dbc-4e7a-ba52-0757ad213aba None None] MFA Rules not processed for user `47296119d0ef495489c4b5806001b954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:58.061488 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a1bb0db-9dbc-4e7a-ba52-0757ad213aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:58.069226 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:58.341926 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:58.343596 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:58.351499 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=65 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:58.359061 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25453-6481-4f17-a8c5-4e6ce588bb84 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:58.362608 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e02e0b0eac04b51970fa87a0df61e64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c5db404946b492d8d06b43ef9e251d3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:58.363256 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:58.363543 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:58.363821 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:58.373008 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:58.374429 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce60600-9ef5-4219-a8fd-55ee5fc37d82 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:58.384017 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 777/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Wed May 25 13:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:51:08.055123 dv-cinder-843278-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:51:08.379648 dv-cinder-843278-1-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}